
Deep Learning at Scale for Gravitational Wave Parameter Estimation of Binary Black Hole Mergers
We present the first application of deep learning at scale to do gravitational wave parameter estimation of binary black hole mergers that describe a 4D signal manifold, i.e., black holes whose spins are aligned or antialigned, and which evolve on quasicircular orbits. We densely sample this 4D signal manifold using over three hundred thousand simulated waveforms. In order to cover a broad range of astrophysically motivated scenarios, we synthetically enhance this waveform dataset to ensure that our deep learning algorithms can process waveforms located at any point in the data stream of gravitational wave detectors (time invariance) for a broad range of signaltonoise ratios (scale invariance), which in turn means that our neural network models are trained with over 10^7 waveform signals. We then apply these neural network models to estimate the astrophysical parameters of black hole mergers, and their corresponding black hole remnants, including the final spin and the gravitational wave quasinormal frequencies. These neural network models represent the first time deep learning is used to provide pointparameter estimation calculations endowed with statistical errors. For each binary black hole merger that groundbased gravitational wave detectors have observed, our deep learning algorithms can reconstruct its parameters within 2 milliseconds using a single Tesla V100 GPU. We show that this new approach produces parameter estimation results that are consistent with Bayesian analyses that have been used to reconstruct the parameters of the catalog of binary black hole mergers observed by the advanced LIGO and Virgo detectors.
03/05/2019 ∙ by Hongyu Shen, et al. ∙ 12 ∙ shareread it

MaxSliced Wasserstein Distance and its use for GANs
Generative adversarial nets (GANs) and variational autoencoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, imagetoimage translation and feature learning. However, to model highdimensional distributions, sequential training and stacked architectures are common, increasing the number of tunable hyperparameters as well as the training time. Nonetheless, the sample complexity of the distance metrics remains one of the factors affecting GAN training. We first show that the recently proposed sliced Wasserstein distance has compelling sample complexity properties when compared to the Wasserstein distance. To further improve the sliced Wasserstein distance we then analyze its `projection complexity' and develop the maxsliced Wasserstein distance which enjoys compelling sample complexity while reducing projection complexity, albeit necessitating a max estimation. We finally illustrate that the proposed distance trains GANs on highdimensional images up to a resolution of 256x256 easily.
04/11/2019 ∙ by Ishan Deshpande, et al. ∙ 12 ∙ shareread it

CryoElectron Microscopy Image Analysis Using MultiFrequency Vector Diffusion Maps
Cryoelectron microscopy (EM) single particle reconstruction is an entirely general technique for 3D structure determination of macromolecular complexes. However, because the images are taken at low electron dose, it is extremely hard to visualize the individual particle with low contrast and high noise level. In this paper, we propose a novel approach called multifrequency vector diffusion maps (MFVDM) to improve the efficiency and accuracy of cryoEM 2D image classification and denoising. This framework incorporates different irreducible representations of the estimated alignment between similar images. In addition, we propose a graph filtering scheme to denoise the images using the eigenvalues and eigenvectors of the MFVDM matrices. Through both simulated and publicly available real data, we demonstrate that our proposed method is efficient and robust to noise compared with the stateoftheart cryoEM 2D class averaging and image restoration algorithms.
04/16/2019 ∙ by Yifeng Fan, et al. ∙ 12 ∙ shareread it

Steerable ePCA
In photonlimited imaging, the pixel intensities are affected by photon count noise. Many applications, such as 3D reconstruction using correlation analysis in Xray free electron laser (XFEL) single molecule imaging, require an accurate estimation of the covariance of the underlying 2D clean images. Accurate estimation of the covariance from lowphoton count images must take into account that pixel intensities are Poisson distributed, rendering the suboptimality of the classical sample covariance estimator. Moreover, in single molecule imaging, including inplane rotated copies of all images could further improve the accuracy of covariance estimation. In this paper we introduce an efficient and accurate algorithm for covariance matrix estimation of count noise 2D images, including their uniform planar rotations and possibly reflection. Our procedure, steerable ePCA, combines in a novel way two recently introduced innovations. The first is a methodology for principal component analysis (PCA) for Poisson distributions, and more generally, exponential family distributions, called ePCA. The second is steerable PCA, a fast and accurate procedure for including all planar rotations for PCA. The resulting principal components are invariant to the rotation and reflection of the input images. We demonstrate the efficiency and accuracy of steerable ePCA in numerical experiments involving simulated XFEL datasets.
12/20/2018 ∙ by Zhizhen Zhao, et al. ∙ 6 ∙ shareread it

Mahalanobis Distance for Class Averaging of CryoEM Images
Single particle reconstruction (SPR) from cryoelectron microscopy (EM) is a technique in which the 3D structure of a molecule needs to be determined from its contrast transfer function (CTF) affected, noisy 2D projection images taken at unknown viewing directions. One of the main challenges in cryoEM is the typically low signal to noise ratio (SNR) of the acquired images. 2D classification of images, followed by class averaging, improves the SNR of the resulting averages, and is used for selecting particles from micrographs and for inspecting the particle images. We introduce a new affinity measure, akin to the Mahalanobis distance, to compare cryoEM images belonging to different defocus groups. The new similarity measure is employed to detect similar images, thereby leading to an improved algorithm for class averaging. We evaluate the performance of the proposed class averaging procedure on synthetic datasets, obtaining state of the art classification.
11/10/2016 ∙ by Tejal Bhamre, et al. ∙ 0 ∙ shareread it

Fast Steerable Principal Component Analysis
Cryoelectron microscopy nowadays often requires the analysis of hundreds of thousands of 2D images as large as a few hundred pixels in each direction. Here we introduce an algorithm that efficiently and accurately performs principal component analysis (PCA) for a large set of twodimensional images, and, for each image, the set of its uniform rotations in the plane and their reflections. For a dataset consisting of n images of size L × L pixels, the computational complexity of our algorithm is O(nL^3 + L^4), while existing algorithms take O(nL^4). The new algorithm computes the expansion coefficients of the images in a FourierBessel basis efficiently using the nonuniform fast Fourier transform. We compare the accuracy and efficiency of the new algorithm with traditional PCA and existing algorithms for steerable PCA.
12/02/2014 ∙ by Zhizhen Zhao, et al. ∙ 0 ∙ shareread it

Rotationally Invariant Image Representation for Viewing Direction Classification in CryoEM
We introduce a new rotationally invariant viewing angle classification method for identifying, among a large number of CryoEM projection images, similar views without prior knowledge of the molecule. Our rotationally invariant features are based on the bispectrum. Each image is denoised and compressed using steerable principal component analysis (PCA) such that rotating an image is equivalent to phase shifting the expansion coefficients. Thus we are able to extend the theory of bispectrum of 1D periodic signals to 2D images. The randomized PCA algorithm is then used to efficiently reduce the dimensionality of the bispectrum coefficients, enabling fast computation of the similarity between any pair of images. The nearest neighbors provide an initial classification of similar viewing angles. In this way, rotational alignment is only performed for images with their nearest neighbors. The initial nearest neighbor classification and alignment are further improved by a new classification method called vector diffusion maps. Our pipeline for viewing angle classification and alignment is experimentally shown to be faster and more accurate than referencefree alignment with rotationally invariant Kmeans clustering, MSA/MRA 2D classification, and their modern approximations.
09/29/2013 ∙ by Zhizhen Zhao, et al. ∙ 0 ∙ shareread it

FourierBessel rotational invariant eigenimages
We present an efficient and accurate algorithm for principal component analysis (PCA) of a large set of two dimensional images, and, for each image, the set of its uniform rotations in the plane and its reflection. The algorithm starts by expanding each image, originally given on a Cartesian grid, in the FourierBessel basis for the disk. Because the images are bandlimited in the Fourier domain, we use a sampling criterion to truncate the FourierBessel expansion such that the maximum amount of information is preserved without the effect of aliasing. The constructed covariance matrix is invariant to rotation and reflection and has a special block diagonal structure. PCA is efficiently done for each block separately. This FourierBessel based PCA detects more meaningful eigenimages and has improved denoising capability compared to traditional PCA for a finite number of noisy images.
11/08/2012 ∙ by Zhizhen Zhao, et al. ∙ 0 ∙ shareread it

Denoising Gravitational Waves using Deep Learning with Recurrent Denoising Autoencoders
Gravitational wave astronomy is a rapidly growing field of modern astrophysics, with observations being made frequently by the LIGO detectors. Gravitational wave signals are often extremely weak and the data from the detectors, such as LIGO, is contaminated with nonGaussian and nonstationary noise, often containing transient disturbances which can obscure real signals. Traditional denoising methods, such as principal component analysis and dictionary learning, are not optimal for dealing with this nonGaussian noise, especially for low signaltonoise ratio gravitational wave signals. Furthermore, these methods are computationally expensive on large datasets. To overcome these issues, we apply stateoftheart signal processing techniques, based on recent groundbreaking advancements in deep learning, to denoise gravitational wave signals embedded either in Gaussian noise or in real LIGO noise. We introduce SMTDAE, a Staired MultiTimestep Denoising Autoencoder, based on sequencetosequence bidirectional LongShortTermMemory recurrent neural networks. We demonstrate the advantages of using our unsupervised deep learning approach and show that, after training only using simulated Gaussian noise, SMTDAE achieves superior recovery performance for gravitational wave signals embedded in real nonGaussian LIGO noise.
11/27/2017 ∙ by Hongyu Shen, et al. ∙ 0 ∙ shareread it

Realtime regression analysis with deep convolutional neural networks
We discuss the development of novel deep learning algorithms to enable realtime regression analysis for time series data. We showcase the application of this new method with a timely case study, and then discuss the applicability of this approach to tackle similar challenges across science domains.
05/07/2018 ∙ by E. A. Huerta, et al. ∙ 0 ∙ shareread it

LanczosNet: MultiScale Deep Graph Convolutional Networks
We propose the Lanczos network (LanczosNet), which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. Relying on the tridiagonal decomposition of the Lanczos algorithm, we not only efficiently exploit multiscale information via fast approximated computation of matrix power but also design learnable spectral filters. Being fully differentiable, LanczosNet facilitates both graph kernel learning as well as learning node embeddings. We show the connection between our LanczosNet and graph based manifold learning methods, especially the diffusion maps. We benchmark our model against several recent deep graph networks on citation networks and QM8 quantum chemistry dataset. Experimental results show that our model achieves the stateoftheart performance in most tasks.
01/06/2019 ∙ by Renjie Liao, et al. ∙ 0 ∙ shareread it
Zhizhen Zhao
is this you? claim profile