Semi-supervised learning of images with strong rotational disorder: assembling nanoparticle libraries

by   Maxim Ziatdinov, et al.

The proliferation of optical, electron, and scanning probe microscopies gives rise to large volumes of imaging data of objects as diversified as cells, bacteria, pollen, to nanoparticles and atoms and molecules. In most cases, the experimental data streams contain images having arbitrary rotations and translations within the image. At the same time, for many cases, small amounts of labeled data are available in the form of prior published results, image collections, and catalogs, or even theoretical models. Here we develop an approach that allows generalizing from a small subset of labeled data with a weak orientational disorder to a large unlabeled dataset with a much stronger orientational (and positional) disorder, i.e., it performs a classification of image data given a small number of examples even in the presence of a distribution shift between the labeled and unlabeled parts. This approach is based on the semi-supervised rotationally invariant variational autoencoder (ss-rVAE) model consisting of the encoder-decoder "block" that learns a rotationally (and translationally) invariant continuous latent representation of data and a classifier that encodes data into a finite number of discrete classes. The classifier part of the trained ss-rVAE inherits the rotational (and translational) invariances and can be deployed independently of the other parts of the model. The performance of the ss-rVAE is illustrated using the synthetic data sets with known factors of variation. We further demonstrate its application for experimental data sets of nanoparticles, creating nanoparticle libraries and disentangling the representations defining the physical factors of variation in the data. The code reproducing the results is available at



There are no comments yet.


page 5

page 6

page 7

page 9

page 10


Semi-Supervised Learning with Self-Supervised Networks

Recent advances in semi-supervised learning have shown tremendous potent...

Semi-Supervised Disentanglement of Class-Related and Class-Independent Factors in VAE

In recent years, extending variational autoencoder's framework to learn ...

VideoSSL: Semi-Supervised Learning for Video Classification

We propose a semi-supervised learning approach for video classification,...

Robust Feature Disentanglement in Imaging Data via Joint Invariant Variational Autoencoders: from Cards to Atoms

Recent advances in imaging from celestial objects in astronomy visualize...

Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders

Recent advances in scanning tunneling and transmission electron microsco...

A Semi-Supervised Generative Adversarial Network for Prediction of Genetic Disease Outcomes

For most diseases, building large databases of labeled genetic data is a...

Decoding the shift-invariant data: applications for band-excitation scanning probe microscopy

A shift-invariant variational autoencoder (shift-VAE) is developed as an...

Code Repositories


Semi-supervised VAE for data with the rotational disorder

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.