Image recovery from rotational and translational invariants

I Introduction

I-a Problem statement

We consider the problem of estimating a target image from a large

m \times m

noisy measurement

M

that contains

p

randomly rotated copies of the target. More precisely, suppose that

f : R^{2} \to R

is a target image supported on the open unit disc, and

f_{ϕ}

is the rotation of

f

by angle

ϕ

about the origin. Let

F_{ϕ} : Z^{2} \to R

be a discrete version of

f_{ϕ}

defined by

F_{ϕ} (\to x) := f_{ϕ} (\to x / n), for \to x \in Z^{2},

(1)

where $n ≪ m$ is a fixed positive integer. We assume the measurement $M : {1, \dots, m}^{2} \to R$ is of the form

M (\to x) := p \sum j = 1 F_{ϕ_{j}} (\to x - {\to x}_{j}) + ε (\to x), for \to x \in {1, \dots, m}^{2},

(2)

where $ϕ_{1}, \dots, ϕ_{p} \in [0, 2 π)$ are uniformly random angles, ${\to x}_{1}, \dots, {\to x}_{p} \in {n, \dots, m - n + 1}^{2}$ are translations of the target images, and $ε (\to x)$ is i.i.d. Gaussian noise with mean $0$

and variance

σ^{2}

. For simplicity, we assume that images in the measurement are separated by at least one image diameter, see (5). Recent work on a related problem suggests that this separation restriction can be alleviated [13].

Our goal is to recover the image $f$ from the noisy measurement $M$ . We emphasize that with respect to this goal, the rotations $ϕ_{1}, \dots, ϕ_{p}$ and translations ${\to x}_{1}, \dots, {\to x}_{p}$ are nuisance parameters that do not necessarily need to be estimated. If the signal-to-noise ratio (SNR) is high as in Figure 1(a), then estimating these rotations and translations is straightforward, and the image can be recovered by aligning and averaging its different copies in the measurement. However, if the SNR is low as in Figure 1(b), which is the case of interest in this study, then this approach may be problematic as even detecting the image occurrences becomes challenging [1, 4].

Fig. 1: Two measurements $M$ with the same five rotated copies of a target image, but different noise levels: SNR = 10 (left), and SNR = 0.1 (right).

We are interested in the following question: given a large enough measurement $M$ , can we recover the target image $f$ regardless of the level of noise? More precisely, if the number of target image occurrences $p$ grows at a constant rate $γ$ with the size $m^{2}$ of the measurement $M$ , can we recover the image $f$ for any fixed noise variance $σ^{2}$ as $m \to \infty$ ?

I-B Motivation

We are motivated by challenges in single-particle reconstruction using cryo-electron microscopy (cryo-EM). The measurements in cryo-EM consist of 2-D tomographic projections of identical biomolecules of unknown 3-D orientations embedded in a large noisy image, called a micrograph. In the current analysis workflow of cryo-EM data [2, 7, 8, 17, 19, 20], the 2-D projections are first detected and extracted from the micrograph, and later rotationally and translationally aligned to reconstruct the 3-D molecular structure. This approach is problematic for small molecules, which are difficult to detect due to their lower SNR. This difficulty of detection in turn sets a lower bound on the usable molecule size in the current analysis workflow of cryo-EM data [9]. To circumvent this fundamental barrier, recent papers [3, 4] suggest to directly estimate the 3-D structure from the micrograph, without an intermediate detection stage; this approach was inspired by Kam [11] who introduced autocorrelation analysis to cryo-EM.

The estimation problem described in Section I-A is a simplified version of the cryo-EM reconstruction problem: the tomographic projection operator is omitted and we observe the same 2-D image multiple times with random in-plane rotations. This image recovery problem is an instance of the multi-target detection statistical model, in which a set of signals appear multiple times at unknown locations in a noisy measurement [3, 4, 13]. Here, we extend previous works by taking in-plane rotations into account, which forms an important step towards the analysis of the full cryo-EM problem.

I-C Main contribution

The principal contribution of this paper is to demonstrate that a target image $f$ can be recovered from a measurement $M$ of the form (2) without estimating the rotations and translations of the images in the measurement. As a consequence, limits on estimating these nuisance parameters do not impose limits on estimating the target image $f$ . In particular, we empirically demonstrate that the estimation of $f$ is feasible at any noise level given a large enough measurement. From a computational perspective, this approach is highly efficient as it requires only one pass over the measurement to calculate the autocorrelations; this is in contrast to the likelihood-based techniques that may become prohibitive as the data size increases. For a more detailed discussion, see [13].

Ii Rotationally and translationally invariant features

We begin by studying invariance in the continuous setting. The simplest invariant to both rotations and translations is

s_{1} := \int_{R^{2}} f (\to x) d \to x,

which is the mean of the image; however, clearly more information is needed to recover the image. Motivated by autocorrelation analysis we next consider the rotationally-averaged second-order autocorrelation $s_{2} : R^{2} \to R$ by

s_{2} ({\to x}_{1}) := \int_{0}^{2 π} \int_{R^{2}} f_{ϕ} (\to x) f_{ϕ} (\to x + {\to x}_{1}) d \to x d ϕ .

By a change of variables of integration, it is clear that $s_{2}$ is only a function of $| {\to x}_{1} |$ , that is, it is a 1-D function, and thus does not contain enough information to encode a 2-D image. Thus, we proceed to consider the rotationally-averaged third-order autocorrelation $s_{3} : R^{2} \times R^{2} \to R$ defined by

s_{3} ({\to x}_{1}, {\to x}_{2}) := \int_{0}^{2 π} \int_{R^{2}} f_{ϕ} (\to x) f_{ϕ} (\to x + {\to x}_{1}) f_{ϕ} (\to x + {\to x}_{2}) d \to x d ϕ .

It is straightforward to verify that $s_{3}$ is a function of three parameters: $| {\to x}_{1} |$ , $| {\to x}_{2} |$ and $θ ({\to x}_{1}, {\to x}_{2})$ , where $θ ({\to x}_{1}, {\to x}_{2})$

denotes the angle between the vectors

{\to x}_{1}, {\to x}_{2} \in R^{2}

. In this work, we show empirically that

f

can be recovered from

S_{3} : Z^{2} \times Z^{2} \to R

, which is a discrete version of

s_{3}

defined by

S_{3} ({\to x}_{1}, {\to x}_{2}) := \int_{0}^{2 π} \sum \to x \in Z^{2} F_{ϕ} (\to x) F_{ϕ} (\to x + {\to x}_{1}) F_{ϕ} (\to x + {\to x}_{2}) d ϕ .

(3)

We claim that the function $S_{3}$ can be approximated from the measurement $M$ by computing the third-order autocorrelation $A_{3} : Z^{2} \times Z^{2} \to R$ defined by

A_{3} ({\to x}_{1}, {\to x}_{2}) := \frac{1}{m^{2}} \sum \to x \in Z^{2} M (\to x) M (\to x + {\to x}_{1}) M (\to x + {\to x}_{2}) .

(4)

Indeed, recall that the images are assumed to be separated in the measurement by one image diameter:

| {\to x}_{j_{1}} - {\to x}_{j_{2}} | > 4 n, for j_{1} \neq j_{2},

(5)

and that the rotations in the measurement are chosen uniformly at random. Under these assumptions it is straightforward to show, see for example [3], that if $p / m^{2} \to γ$ , then

A_{3} ({\to x}_{1}, {\to x}_{2}) \to \frac{γ}{2 π} S_{3} ({\to x}_{1}, {\to x}_{2}) + σ^{2} \frac{γ S_{1}}{2 π} (δ (_{1}) + δ (_{2}) + δ (_{1} -_{2})),

(6)

as $m \to \infty$ for any fixed noise level $σ^{2}$ , where

S_{1} := \int_{0}^{2 π} \sum \to x \in Z^{2} F_{ϕ} (\to x) d ϕ,

and $δ (\to x) = 1$ if $\to x = \to 0$ and is zero otherwise. Crucially, equation (6) relates $A_{3}$ , a quantity that can be estimated accurately from the data, with functions of the target: $S_{1}$ and $S_{3}$ , while averaging out the effects of the nuisance variables. In practice, $σ^{2}$ and $γ S_{1}$ can be estimated from $M$ . More specifically, $σ^{2}$ can be estimated by the variance of the pixel values of $M$ in the low SNR regime, while $γ S_{1}$ can be estimated by the empirical mean of $M$ . As a result, $S_{3}$ can be estimated from $A_{3}$ up to a constant factor $γ$ .

The analysis above motivates the following question: can the image $f$ be robustly reconstructed from $S_{3}$ ? We demonstrate empirically that the answer to this question is positive. For simplicity of exposition, we describe a method of recovering $f$ that just involves $S_{3}$ , although we note that $S_{1}$ and $S_{2}$ , which represent the discrete analogs of $s_{1}$ and $s_{2}$ , may be used to aid in the reconstruction process.

The problem of estimating $f$ from $M$ , as described in Section I-A, should not be confused with the related problem of recovering an image from a set of measurements ${M_{j}}_{j = 1}^{n}$

, each of which has exactly one shifted and rotated observation. Several moment-based techniques have been proposed to address this problem at high noise levels, see for instance

[12, 15, 16, 18], but none can be applied directly to our problem where only a single, large observation

M

is available.

Iii Image recovery from invariants

Iii-a Steerable basis

Recall that the target image $f : R^{2} \to R$ is supported on the unit disc. We assume that $f$

is bandlimited in the basis of Dirichlet Laplacian eigenfunctions on the unit disk. In polar coordinates

(r, θ)

, these eigenfunctions are of the form

ψ_{ν, q} (r, θ) := J_{ν} (λ_{ν, q} r) e^{i ν θ}, for (ν, q) \in Z \times Z_{> 0},

(7)

where $J_{ν}$ is the $ν$ -th order Bessel function of the first kind, and $λ_{ν, q} > 0$ is the $q$ -th positive root of $J_{ν}$

. The eigenvalue associated with

ψ_{ν, q}

λ_{ν, q}^{2}

, and thus, the assumption that

f

is bandlimited can be written as

f (r, θ) = \sum (ν, q) : λ_{ν, q} \leq λ α_{ν, q} ψ_{ν, q} (r, θ), for r \leq 1,

(8)

where $λ > 0$ is the bandlimit frequency, and $α_{ν, q}$ are the associated expansion coefficients. If we further define

g_{ν} (r, θ) = \sum q : λ_{ν, q} \leq λ α_{ν, q} ψ_{ν, q} (r, θ),

then we can write

f (r, θ) = ν_{m a x} \sum ν = - ν_{m a x} g_{ν} (r, θ),

(9)

where $ν_{max} := max {ν : λ_{ν, 1} \leq λ}$ . An advantage of expressing a function in Dirichlet Laplacian eigenfunctions is that the basis is steerable in the sense that it diagonalizes the rotation operator: rotating $f$ by $ϕ$ corresponds to multiplying each term of $g_{ν}$ by $e^{i ν ϕ}$ :

f (r, θ + ϕ) = ν_{m a x} \sum ν = - ν_{m a x} g_{ν} (r, θ) e^{i ν ϕ} .

(10)

Iii-B Computing invariants

Recall that $F_{ϕ} : Z^{2} \to R$ is the discretely sampled version of $f_{ϕ}$ , which is defined by $F_{ϕ} (\to x) = f_{ϕ} (\to x / n)$ for $\to x \in Z^{2}$ . In the following, we consider $F_{ϕ}$ as a function on

J := {- 2 n, \dots, 2 n - 1}^{2} \subset Z^{2} .

Since $f_{ϕ}$ is supported on the open unit disc, it follows that

supp (F_{ϕ}) \subset {\to x \in J : | \to x | < n} .

Let ${^F}_{ϕ} : J \to C$ by

{^F}_{ϕ} (\to k) = \sum \to x \in J F_{ϕ} (\to x) e^{- 2 π i \to k \cdot \to x / 4 n}, for \to k \in J,

denote the discrete Fourier transform (DFT) of

F_{ϕ}

. Similarly, we can consider

S_{3}

as a function on

J \times J

and define its DFT

{^S}_{3} : J \times J \to C

(11)

for ${\to k}_{1}, {\to k}_{2} \in J$ . By substituting (3) into (11), it is straightforward to show that

{^S}_{3} ({\to k}_{1}, {\to k}_{2}) = \int_{0}^{2 π} {^F}_{ϕ} ({\to k}_{1}) {^F}_{ϕ} ({\to k}_{2}) {^F}_{ϕ} (- {\to k}_{1} - {\to k}_{2}) d ϕ .

(12)

The triple product in (12) corresponds to the Fourier transform of the third-order autocorrelation. This triple product is called the bispectrum [21] and many of its analytical and computational properties have been studied; see for instance [5, 18]. We note that the bispectrum can be generalized to more involved operations, such as 2-D rotations [14], 3-D rotations [12], and general compact groups [10].

Define $Ψ_{ν, q} : J \to C$ as the discrete samples of the Dirichlet Laplacian eigenfunctions:

Ψ_{ν, q} (\to x) = ψ_{ν, q} (\to x / n), for \to x \in J,

where $ψ_{ν, q}$ is considered as a function supported on the unit disc. With this notation, we can express $F_{ϕ}$ as

	$F_{ϕ} (\to x)$	$= \sum (ν, q) : λ_{ν, q} \leq λ α_{ν, q} Ψ_{ν, q} (\to x) e^{i ν ϕ}$
		$= ν_{m a x} \sum ν = - ν_{m a x} ⎛ ⎝ \sum q : λ_{ν, q} \leq λ α_{ν, q} Ψ_{ν, q} (\to x) ⎞ ⎠ e^{i ν ϕ} .$

As a consequence, the products $F_{ϕ} (\to x) F_{ϕ} (\to x + {\to x}_{1}) F_{ϕ} (\to x + {\to x}_{2})$ that appear in (3) are bandlimited by $3 ν_{m a x}$ with respect to $ϕ$ . Therefore, we can replace the integral over $ϕ$ in (12) by a summation over angles sampled at the Nyquist rate, that is,

{^S}_{3} ({\to k}_{1}, {\to k}_{2}) = 6 ν_{max} - 1 \sum ν = 0 {^F}_{ϕ_{ν}} ({\to k}_{1}) {^F}_{ϕ_{ν}} ({\to k}_{2}) {^F}_{ϕ_{ν}} (- {\to k}_{1} - {\to k}_{2}),

where $ϕ_{ν} := 2 π ν / (6 ν_{m a x})$ . By linearity of the DFT, we have

{^F}_{ϕ} (\to k) = \sum (ν, q) : λ_{ν, q} \leq λ α_{ν, q} {^Ψ}_{ν, q} (\to k) e^{i ν ϕ},

where ${^Ψ}_{ν, q} : J \to C$ denotes the DFT of $Ψ_{ν, q} : J \to C$ .

Let $V$ denote the set of all the pairs $(ν, q)$ in the expansion above. We define the column vector $w_{\to k, ϕ} \in C^{V}$ by $(w_{\to k, ϕ})_{ν, q} = {^Ψ}_{ν, q} (\to k) e^{i ν ϕ}$ , and the column vector $z \in C^{V}$ by $(z)_{ν, q} = α_{ν, q}$ ; the latter encodes the parameters that describe the target image. With this notation, ${^F}_{ϕ} (\to k)$ can be expressed compactly as ${^F}_{ϕ} (\to k) = z^{⊤} w_{\to k, ϕ},$ and it follows that

{^S}_{3}^{z} ({\to k}_{1}, {\to k}_{2}) = 6 ν_{m a x} - 1 \sum ν = 0 (z^{⊤} w_{{\to k}_{1}, ϕ_{ν}}) (z^{⊤} w_{{\to k}_{2}, ϕ_{ν}}) (z^{⊤} w_{- {\to k}_{1} - {\to k}_{2}, ϕ_{ν}}),

(13)

where we write ${^S}_{3}^{z}$ to emphasize the dependence on $z$ . This expression for ${^S}_{3}^{z}$ is particularly convenient for computational purposes. In particular, the gradient of ${^S}_{3}^{z} ({\to k}_{1}, {\to k}_{2})$ with respect to $z$ is easy to compute, which is important for solving the optimization problem defined in Section III-D.

Iii-C Leveraging symmetries

Recall that $S_{3} ({\to x}_{1}, {\to x}_{2})$ is a discrete version of $s_{3} ({\to x}_{1}, {\to x}_{2})$ , which only depends on the three parameters: $| {\to x}_{1} |$ , $| {\to x}_{2} |$ and $θ ({\to x}_{1}, {\to x}_{2})$ . Let $S_{3}^{*}$ be the estimate of $S_{3}$ from the noisy measurement $M$ via (6), that is, the de-biased and normalized autocorrelation of the measurement $A_{3}$ (4), and ${^S}_{3}^{*}$ denote the DFT of $S_{3}^{*}$ . Since the noise is assumed to be i.i.d. Gaussian with mean $0$ , its Fourier transform is also i.i.d. Gaussian with mean $0$ in Fourier space. Therefore, we can reduce the effect of noise by binning the entries of ${^S}_{3}^{*}$ with similar values of $| {\to k}_{1} |$ , $| {\to k}_{2} |$ and $θ ({\to k}_{1}, {\to k}_{2})$ . Numerically, the binning is done through the mapping $B : J \times J \to I$ by

B ({\to k}_{1}, {\to k}_{2}) = (⌊ b_{1} | {\to k}_{1} | ⌋, ⌊ b_{1} | {\to k}_{2} | ⌋, ⌊ b_{2} θ ({\to k}_{1}, {\to k}_{2}) ⌋),

where $I \subset Z_{\geq 0}^{3}$ represents the set of the bins, and $b_{1}, b_{2} \in R$ determine the bin sizes.

Iii-D Optimization problem

Suppose that we are given an estimate of ${^S}_{3}^{*}$ —the de-biased and normalized third-order autocorrelation of the measurement. For a fixed model and any coefficient vector $z \in C^{V}$ , we can compute ${^S}_{3}^{z}$ via (13). In order to estimate the coefficient vector that is consistent with ${^S}_{3}^{*}$ , we define the cost function

f (z) := \sum \to j \in I {⎛ ⎜ ⎝ \sum ({\to k}_{1}, {\to k}_{2}) : B ({\to k}_{1}, {\to k}_{2}) = \to j {^S}_{3}^{z} ({\to k}_{1}, {\to k}_{2}) - {^S}_{3}^{*} ({\to k}_{1}, {\to k}_{2}) ⎞ ⎟ ⎠}_{1}^{2} .

(14)

Minimizing this cost function is a non-convex (polynomial of degree $6$ ) least squares optimization problem, and thus, a priori, there is no reason to suspect that the global minimum of this problem can be attained; however, our numerical results indicate that standard gradient-based methods result in accurate and stable recovery of the parameters $z$ . We note that computing the cost and gradient involves $O (n^{4} ν_{max})$ operations in each iteration of the optimization.

Iv Numerical results

We first consider the problem of recovering a model image from a noiseless set of ${^S}_{3}^{*}$ —namely, from the exact rotationally-averaged third-order autocorrelation. This reflects the case where the noise level is fixed and $m \to \infty$ . The model image is generated by expanding a $65 \times 65$ image of a tiger (Figure 2) into a linear combination of the first $600$ Dirichlet Laplacian eigenfunctions, sorted by the eigenvalues in ascending order. We denote this model image by $F_{0}$ . Since ${^S}_{3}^{*}$ is noiseless in this case, we accelerate the optimization by only choosing one entry from each bin in $I$ instead of summing all the entries to calculate the cost and gradient, which reduces the computational cost by $O (n)$ .

Let $F_{z}$ be the image formed by the coefficients $z$ that were recovered by minimizing the least squares (14). Since ${^S}_{3}^{*}$ is invariant under in-plane rotations of the model image, we only expect to reconstruct the image $F_{0}$ up to some arbitrary rotation $ϕ$ . As a result, we define the reconstruction error by

{error}_{recon} := inf ϕ \in [0, 2 π) \frac{∥ F_{0} - F_{z}^{ϕ} ∥_{2}}{∥ F_{0} ∥_{2}},

where $F_{z}^{ϕ}$ is the rotation of $F_{z}$ by an angle $ϕ$ . Using the BFGS optimization algorithm, we recovered the image in Figure 2 with ${error}_{recon} = 5 \times 10^{- 12}$ . The optimization took $6.5 \times 10^{4}$ seconds parallelized over 100 CPUs in total.

Fig. 2: A $65 \times 65$ image of a tiger expanded in the first $600$ eigenfunctions, which can be recovered from a noiseless set of ${^S}_{3}^{*}$ to high precision.

In the second experiment, we study the robustness of the reconstruction to noise. Limited by the computational resources, we downsample the image of tiger to $17 \times 17$ pixels, and expand the image into the first $100$ Dirichlet Laplacian eigenfunctions. From the expansion we compute the rotationally-averaged third-order autocorrelation $S_{3}$ , which is further contaminated by i.i.d. Gaussian noise with mean $0$ and variance $σ^{2}$ . A total of 10 different values of $σ$ are used to model the noisy counterparts $S_{3}^{*}$ estimated from measurements of different SNR. The relative error of $S_{3}^{*}$ is quantified by

{error}_{S_{3}^{*}} := \frac{∥ S_{3} - S_{3}^{*} ∥_{2}}{∥ S_{3} ∥_{2}} .

The relative errors of the reconstructed images from the 10 sets of ${^S}_{3}^{*}$ are shown in Figure 3. We observe that the images can be reliably recovered over a wide range of noise levels. The high correlation between ${error}_{S_{3}^{*}}$ and ${error}_{recon}$ might indicate that the optimization landscape is benign.

errorS∗3errorrecon1.9×10−41.6×10−33.9×10−43.8×10−37.7×10−44.9×10−31.5×10−31.0×10−23.1×10−32.3×10−26.2×10−34.5×10−21.2×10−21.1×10−12.4×10−21.5×10−15.0×10−23.2×10−19.9×10−26.9×10−1\raisebox−0.525pt\includegraphics[width=86.724pt]fig03−eps−converted−to.pdf

Fig. 3: We consider

S_{3}^{*}

with

10

different levels of Gaussian noise quantified by

{error}_{S_{3}^{*}}

. For each level of noise, we run the optimization five times with random initializations; we report

{error}_{recon}

corresponding to the trial that minimizes the cost function (14); this procedure can be preformed in practice and makes the results highly consistent. The slope of the best-fit line is

0.97

V Discussion

This work serves as a proof of concept for the feasibility of estimating a target image from its rotational and translational invariants. We have demonstrated the reconstruction of a target image from its noiseless invariants, and showed that our algorithmic approach is robust to noise. In future work, we intend to extend the framework to include the recovery of the target image from the observation $M$ , to mitigate the separation condition [13], and to allow the recovery of multiple images simultaneously, in a similar fashion to [4, 6]. From a theoretical perspective, we wish to complement the empirical results of this work by proving that indeed a generic image $f$ is determined uniquely by its rotationally-averaged third-order autocorrelation.

Acknowledgment

The authors thank Iris Rukshin for stimulating discussions. NFM was supported by NSF DMS-1903015. TYL and AS were supported in part by Award Number FA9550-17-1-0291 from AFOSR, Simons Foundation Math+X Investigator Award, the Moore Foundation Data-Driven Discovery Investigator Award, and NSF BIGDATA Award IIS-1837992.

References

[1] C. Aguerrebere, M. Delbracio, A. Bartesaghi, and G. Sapiro, “Fundamental limits in multi-image alignment,” IEEE Transactions on Signal Processing, vol. 64, no. 21, pp. 5707–5722, 2016.
[2] T. Bendory, A. Bartesaghi, and A. Singer, “Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities”, arXiv preprint arXiv:1908.00574, 2019.
[3] T. Bendory, N. Boumal, W. Leeb, E. Levin, and A. Singer, “Toward single particle reconstruction without particle picking: Breaking the detection limit,” arXiv preprint arXiv:1810.00226, 2018.
[4] T. Bendory, N. Boumal, W. Leeb, E. Levin, and A. Singer, “Multi-target detection with application to cryo-electron microscopy,” Inverse Problems, 2019.
[5] T. Bendory, N. Boumal, C. Ma, Z. Zhao, and A. Singer, “Bispectrum inversion with application to multireference alignment,” IEEE Transactions on Signal Processing, vol. 66, no. 4, pp. 1037–1050, 2017.
[6] N. Boumal, T. Bendory, R. R. Lederman, and A. Singer, “Heterogeneous multireference alignment: A single pass approach,” in 2018 52nd Annual Conference on Information Sciences and Systems (CISS). IEEE, 2018, pp. 1–6.
[7] J. Frank, Three-dimensional electron microscopy of macromolecular assemblies: visualization of biological molecules in their native state. Oxford University Press, 2006.
[8] T. Grant, A. Rohou, and N. Grigorieff, “cisTEM, user-friendly software for single-particle image processing,” Elife, vol. 7, p. e35383, 2018.
[9] R. Henderson, “The potential and limitations of neutrons, electrons and X-rays for atomic resolution microscopy of unstained biological molecules,” Quarterly reviews of biophysics, vol. 28, no. 2, pp. 171–193, 1995.
[10] R. Kakarala, “The bispectrum as a source of phase-sensitive invariants for Fourier descriptors: a group-theoretic approach,” Journal of Mathematical Imaging and Vision, vol. 44, no. 3, pp. 341–353, 2012.
[11] Z. Kam, “The reconstruction of structure from electron micrographs of randomly oriented particles,” Journal of Theoretical Biology, vol. 82, no. 1, pp. 15–39, 1980.
[12] R. Kondor, “A novel set of rotationally and translationally invariant features for images based on the non-commutative bispectrum,” arXiv preprint arXiv:0701127, 2007.
[13] T.-Y. Lan, T. Bendory, N. Boumal, and A. Singer, “Multi-target detection with an arbitrary spacing distribution,” arXiv preprint arXiv:1905.03176, 2019.
[14] C. Ma, T. Bendory, N. Boumal, F. Sigworth, and A. Singer, “Heterogeneous multireference alignment for images with application to 2-D classification in single particle reconstruction,” To appear in IEEE Transactions on Image Processing, 2019.
[15] S. Mallat, “Group invariant scattering”, Communications on Pure and Applied Mathematics vol. 65, no. 10, pp. 1331–1398, 2012.
[16] O. Özyeşil, N. Sharon, and A. Singer, “Synchronization over cartan motion groups via contraction”, SIAM Journal on Applied Algebra and Geometry, vol. 2, no. 2, pp. 207–241, 2018.
[17] A. Punjani, J. L. Rubinstein, D. J. Fleet, and M. A. Brubaker, “cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination,” Nature methods, vol. 14, no. 3, p. 290, 2017.
[18] B. M. Sadler and G. B. Giannakis, “Shift-and rotation-invariant object reconstruction using the bispectrum,” JOSA A, vol. 9, no. 1, pp. 57–69, 1992.
[19] S. H. Scheres, “RELION: implementation of a Bayesian approach to cryo-EM structure determination,” Journal of structural biology, vol. 180, no. 3, pp. 519–530, 2012.
[20] G. Tang, L. Peng, P. R. Baldwin, D. S. Mann, W. Jiang, I. Rees, and S. J. Ludtke, “EMAN2: an extensible image processing suite for electron microscopy,” Journal of structural biology, vol. 157, no. 1, pp. 38–46, 2007.
[21] J. Tukey, “The spectral representation and transformation properties of the higher moments of stationary time series,” Reprinted in The Collected Works of John W. Tukey, vol. 1, pp. 165–184, 1953.

Image recovery from rotational and translational invariants

Adaptive nonparametric detection in cryo-electron microscopy

Multi-target detection with rotations

Cryo-Electron Microscopy Image Analysis Using Multi-Frequency Vector Diffusion Maps

Learning Rotation Invariant Features for Cryogenic Electron Microscopy Image Reconstruction

Estimation under group actions: recovering orbits from invariants

Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities

Wavelet invariants for statistically robust multi-reference alignment

I Introduction

I-a Problem statement

I-B Motivation

I-C Main contribution

Ii Rotationally and translationally invariant features

Iii Image recovery from invariants

Iii-a Steerable basis

Iii-B Computing invariants

Iii-C Leveraging symmetries

Iii-D Optimization problem

Iv Numerical results

V Discussion

Acknowledgment

References

Image recovery from rotational and translational invariants

Related Research

Adaptive nonparametric detection in cryo-electron microscopy

Multi-target detection with rotations

Cryo-Electron Microscopy Image Analysis Using Multi-Frequency Vector Diffusion Maps

Learning Rotation Invariant Features for Cryogenic Electron Microscopy Image Reconstruction

Estimation under group actions: recovering orbits from invariants

Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities

Wavelet invariants for statistically robust multi-reference alignment

I Introduction

I-a Problem statement

I-B Motivation

I-C Main contribution

Ii Rotationally and translationally invariant features

Iii Image recovery from invariants

Iii-a Steerable basis

Iii-B Computing invariants

Iii-C Leveraging symmetries

Iii-D Optimization problem

Iv Numerical results

V Discussion

Acknowledgment

References