Log In Sign Up

Explaining Representation by Mutual Information

by   Lifeng Gu, et al.

Science is used to discover the law of world. Machine learning can be used to discover the law of data. In recent years, there are more and more research about interpretability in machine learning community. We hope the machine learning methods are safe, interpretable, and they can help us to find meaningful pattern in data. In this paper, we focus on interpretability of deep representation. We propose a interpretable method of representation based on mutual information, which summarizes the interpretation of representation into three types of information between input data and representation. We further proposed MI-LR module, which can be inserted into the model to estimate the amount of information to explain the model's representation. Finally, we verify the method through the visualization of the prototype network.


page 6

page 7

page 8

page 9

page 10


Discovering Influential Factors in Variational Autoencoder

In the field of machine learning, it is still a critical issue to identi...

Tensor networks and efficient descriptions of classical data

We investigate the potential of tensor network based machine learning me...

Towards A Rigorous Science of Interpretable Machine Learning

As machine learning systems become ubiquitous, there has been a surge of...

Inverting Supervised Representations with Autoregressive Neural Density Models

Understanding the nature of representations learned by supervised machin...

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric...

Unsupervised machine learning for physical concepts

In recent years, machine learning methods have been used to assist scien...

Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

The attention that deep learning has garnered from the academic communit...