On the interplay between physical and content priors in deep learning for computational imaging

04/14/2020
by   Mo Deng, et al.
0

Deep learning (DL) has been applied extensively in many computational imaging problems, often leading to superior performance over traditional iterative approaches. However, two important questions remain largely unanswered: first, how well can the trained neural network generalize to objects very different from the ones in training? This is particularly important in practice, since large-scale annotated examples similar to those of interest are often not available during training. Second, has the trained neural network learnt the underlying (inverse) physics model, or has it merely done something trivial, such as memorizing the examples or point-wise pattern matching? This pertains to the interpretability of machine-learning based algorithms. In this work, we use the Phase Extraction Neural Network (PhENN), a deep neural network (DNN) for quantitative phase retrieval in a lensless phase imaging system as the standard platform and show that the two questions are related and share a common crux: the choice of the training examples. Moreover, we connect the strength of the regularization effect imposed by a training set to the training process with the Shannon entropy of images in the dataset. That is, the higher the entropy of the training images, the weaker the regularization effect can be imposed. We also discover that weaker regularization effect leads to better learning of the underlying propagation model, i.e. the weak object transfer function, applicable for weakly scattering objects under the weak object approximation. Finally, simulation and experimental results show that better cross-domain generalization performance can be achieved if DNN is trained on a higher-entropy database, e.g. the ImageNet, than if the same DNN is trained on a lower-entropy database, e.g. MNIST, as the former allows the underlying physics model be learned better than the latter.

READ FULL TEXT

page 2

page 7

page 8

page 10

page 12

page 15

page 16

page 19

research
11/13/2021

Physics-guided Loss Functions Improve Deep Learning Performance in Inverse Scattering

Solving electromagnetic inverse scattering problems (ISPs) is challengin...
research
06/16/2020

Real-time 3D Nanoscale Coherent Imaging via Physics-aware Deep Learning

Phase retrieval, the problem of recovering lost phase information from m...
research
07/12/2023

Local Conditional Neural Fields for Versatile and Generalizable Large-Scale Reconstructions in Computational Imaging

Deep learning has transformed computational imaging, but traditional pix...
research
09/24/2020

Robust Phase Unwrapping via Deep Image Prior for Quantitative Phase Imaging

Quantitative phase imaging (QPI) is an emerging label-free technique tha...
research
08/18/2022

Towards Practical Single-shot Phase Retrieval with Physics-Driven Deep Neural Network

Phase retrieval (PR), a long-established challenge for recovering a comp...
research
11/26/2019

Spectra2pix: Generating Nanostructure Images from Spectra

The design of the nanostructures that are used in the field of nano-phot...
research
03/25/2022

Fast fluorescence lifetime imaging analysis via extreme learning machine

We present a fast and accurate analytical method for fluorescence lifeti...

Please sign up or login with your details

Forgot password? Click here to reset