PAC Bayesian Performance Guarantees for Deep (Stochastic) Networks in Medical Imaging

by   Anthony Sicilia, et al.

Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are somehow prone to overfitting and are thus unable to generalize well when datasets are small. The claim is not baseless and likely stems from the observation that PAC bounds on generalization error are usually so large for deep networks that they are vacuous (i.e., logically meaningless). Contrary to this, recent advances using the PAC-Bayesian framework have instead shown non-vacuous bounds on generalization error for large (stochastic) networks and standard datasets (e.g., MNIST and CIFAR-10). We apply these techniques to a much smaller medical imagining dataset (the ISIC 2018 challenge set). Further, we consider generalization of deep networks on segmentation tasks which has not commonly been done using the PAC-Bayesian framework. Importantly, we observe that the resultant bounds are also non-vacuous despite the sharp reduction in sample size. In total, our results demonstrate the applicability of PAC-Bayesian bounds for deep stochastic networks in the medical imaging domain.


page 1

page 2

page 3

page 4


On Margins and Derandomisation in PAC-Bayes

We develop a framework for derandomising PAC-Bayesian generalisation bou...

Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks

We present a comprehensive study of multilayer neural networks with bina...

Imaging Conductivity from Current Density Magnitude using Neural Networks

Conductivity imaging represents one of the most important tasks in medic...

PAC-Bayesian theory for stochastic LTI systems

In this paper we derive a PAC-Bayesian error bound for autonomous stocha...

Does the Data Induce Capacity Control in Deep Learning?

This paper studies how the dataset may be the cause of the anomalous gen...

Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience

The ability of overparameterized deep networks to generalize well has be...

Uniform convergence may be unable to explain generalization in deep learning

We cast doubt on the power of uniform convergence-based generalization b...

Code Repositories


The repo for the pre-print work "PAC Bayesian Performance Guarantees for Deep(Stochastic) Networks in Medical Imaging."

view repo