DeepAI AI Chat
Log In Sign Up

Representation Learning with Information Theory for COVID-19 Detection

by   Abel Díaz Berenguer, et al.

Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, we can conveniently implement strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). The LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory measures like mutual information, known to be computationally expensive due to a density estimation step, our LSMI formulation alleviates the issues of intractable mutual information estimation and can be used to approximate it. Experiments on CT based COVID-19 Detection and COVID-19 Severity Detection benchmarks demonstrate the effectiveness of our method.


page 1

page 2

page 3

page 4


Mutual Information Gradient Estimation for Representation Learning

Mutual Information (MI) plays an important role in representation learni...

Deep Sufficient Representation Learning via Mutual Information

We propose a mutual information-based sufficient representation learning...

Wasserstein Dependency Measure for Representation Learning

Mutual information maximization has emerged as a powerful learning objec...

A Theory of Usable Information Under Computational Constraints

We propose a new framework for reasoning about information in complex sy...

AI-MIA: COVID-19 Detection Severity Analysis through Medical Imaging

This paper presents the baseline approach for the organized 2nd Covid-19...

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric...