DeepAI
Log In Sign Up

Explaining Representation by Mutual Information

03/28/2021
by   Lifeng Gu, et al.
0

Science is used to discover the law of world. Machine learning can be used to discover the law of data. In recent years, there are more and more research about interpretability in machine learning community. We hope the machine learning methods are safe, interpretable, and they can help us to find meaningful pattern in data. In this paper, we focus on interpretability of deep representation. We propose a interpretable method of representation based on mutual information, which summarizes the interpretation of representation into three types of information between input data and representation. We further proposed MI-LR module, which can be inserted into the model to estimate the amount of information to explain the model's representation. Finally, we verify the method through the visualization of the prototype network.

READ FULL TEXT

page 6

page 7

page 8

page 9

page 10

09/06/2018

Discovering Influential Factors in Variational Autoencoder

In the field of machine learning, it is still a critical issue to identi...
03/11/2021

Tensor networks and efficient descriptions of classical data

We investigate the potential of tensor network based machine learning me...
02/28/2017

Towards A Rigorous Science of Interpretable Machine Learning

As machine learning systems become ubiquitous, there has been a surge of...
06/01/2018

Inverting Supervised Representations with Autoregressive Neural Density Models

Understanding the nature of representations learned by supervised machin...
10/31/2022

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric...
05/11/2022

Unsupervised machine learning for physical concepts

In recent years, machine learning methods have been used to assist scien...
09/16/2020

Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

The attention that deep learning has garnered from the academic communit...