PAC Bayesian Performance Guarantees for Deep (Stochastic) Networks in Medical Imaging

04/12/2021
by   Anthony Sicilia, et al.
10

Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are somehow prone to overfitting and are thus unable to generalize well when datasets are small. The claim is not baseless and likely stems from the observation that PAC bounds on generalization error are usually so large for deep networks that they are vacuous (i.e., logically meaningless). Contrary to this, recent advances using the PAC-Bayesian framework have instead shown non-vacuous bounds on generalization error for large (stochastic) networks and standard datasets (e.g., MNIST and CIFAR-10). We apply these techniques to a much smaller medical imagining dataset (the ISIC 2018 challenge set). Further, we consider generalization of deep networks on segmentation tasks which has not commonly been done using the PAC-Bayesian framework. Importantly, we observe that the resultant bounds are also non-vacuous despite the sharp reduction in sample size. In total, our results demonstrate the applicability of PAC-Bayesian bounds for deep stochastic networks in the medical imaging domain.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/08/2021

On Margins and Derandomisation in PAC-Bayes

We develop a framework for derandomising PAC-Bayesian generalisation bou...
05/24/2019

Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks

We present a comprehensive study of multilayer neural networks with bina...
04/05/2022

Imaging Conductivity from Current Density Magnitude using Neural Networks

Conductivity imaging represents one of the most important tasks in medic...
03/23/2021

PAC-Bayesian theory for stochastic LTI systems

In this paper we derive a PAC-Bayesian error bound for autonomous stocha...
10/27/2021

Does the Data Induce Capacity Control in Deep Learning?

This paper studies how the dataset may be the cause of the anomalous gen...
05/30/2019

Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience

The ability of overparameterized deep networks to generalize well has be...
02/13/2019

Uniform convergence may be unable to explain generalization in deep learning

We cast doubt on the power of uniform convergence-based generalization b...

Code Repositories

PAC-Bayes-In-Medical-Imaging

The repo for the pre-print work "PAC Bayesian Performance Guarantees for Deep(Stochastic) Networks in Medical Imaging."


view repo