DeepAI AI Chat
Log In Sign Up

Differentiable probabilistic models of scientific imaging with the Fourier slice theorem

by   Karen Ullrich, et al.

Scientific imaging techniques such as optical and electron microscopy and computed tomography (CT) scanning are used to study the 3D structure of an object through 2D observations. These observations are related to the original 3D object through orthogonal integral projections. For common 3D reconstruction algorithms, computational efficiency requires the modeling of the 3D structures to take place in Fourier space by applying the Fourier slice theorem. At present, it is unclear how to differentiate through the projection operator, and hence current learning algorithms can not rely on gradient based methods to optimize 3D structure models. In this paper we show how back-propagation through the projection operator in Fourier space can be achieved. We demonstrate the validity of the approach with experiments on 3D reconstruction of proteins. We further extend our approach to learning probabilistic models of 3D objects. This allows us to predict regions of low sampling rates or estimate noise. A higher sample efficiency can be reached by utilizing the learned uncertainties of the 3D structure as an unsupervised estimate of the model fit. Finally, we demonstrate how the reconstruction algorithm can be extended with an amortized inference scheme on unknown attributes such as object pose. Through empirical studies we show that joint inference of the 3D structure and the object pose becomes more difficult when the ground truth object contains more symmetries. Due to the presence of for instance (approximate) rotational symmetries, the pose estimation can easily get stuck in local optima, inhibiting a fine-grained high-quality estimate of the 3D structure.


page 1

page 3

page 8

page 9

page 12

page 13


A Local Fourier Slice Theorem

We present a local Fourier slice equation that enables local and sparse ...

Accelerated FBP for computed tomography image reconstruction

Filtered back projection (FBP) is a commonly used technique in tomograph...

Motion Detection in Diffraction Tomography by Common Circle Methods

The method of common lines is a well-established reconstruction techniqu...

A Bayesian Approach to CT Reconstruction with Uncertain Geometry

Computed tomography is a method for synthesizing volumetric or cross-sec...

Masked Sinogram Model with Transformer for ill-Posed Computed Tomography Reconstruction: a Preliminary Study

Computed Tomography (CT) is an imaging technique where information about...

CNN-based regularisation for CT image reconstructions

X-ray computed tomographic infrastructures are medical imaging modalitie...

Local Fourier Slice Photography

Light field cameras provide intriguing possibilities, such as post-captu...

Code Repositories


This code accompanies "Differentiable probabilistic models of scientific imaging with the Fourier slice theorem", UAI 2019

view repo