Using CycleGANs to Generate Realistic STEM Images for Machine Learning

01/18/2023
by   Abid Khan, et al.
0

The rise of automation and machine learning (ML) in electron microscopy has the potential to revolutionize materials research by enabling the autonomous collection and processing of vast amounts of atomic resolution data. However, a major challenge is developing ML models that can reliably and rapidly generalize to large data sets with varying experimental conditions. To overcome this challenge, we develop a cycle generative adversarial network (CycleGAN) that introduces a novel reciprocal space discriminator to augment simulated data with realistic, complex spatial frequency information learned from experimental data. This enables the CycleGAN to generate nearly indistinguishable images from real experimental data, while also providing labels for further ML applications. We demonstrate the effectiveness of this approach by training a fully convolutional network (FCN) to identify single atom defects in a large data set of 4.5 million atoms, which we collected using automated acquisition in an aberration-corrected scanning transmission electron microscope (STEM). Our approach yields highly adaptable FCNs that can adjust to dynamically changing experimental variables, such as lens aberrations, noise, and local contamination, with minimal manual intervention. This represents a significant step towards building fully autonomous approaches for harnessing microscopy big data.

READ FULL TEXT

page 17

page 18

page 19

page 20

page 21

page 31

page 32

research
06/09/2022

STEM image analysis based on deep learning: identification of vacancy defects and polymorphs of MoS_2

Scanning transmission electron microscopy (STEM) is an indispensable too...
research
09/12/2023

A robust synthetic data generation framework for machine learning in High-Resolution Transmission Electron Microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly...
research
04/04/2023

Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy

Machine learning (ML) has become critical for post-acquisition data anal...
research
03/22/2021

Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becomin...
research
08/05/2020

Machine learning for faster and smarter fluorescence lifetime imaging microscopy

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique ...
research
01/09/2022

Machine learning enabling high-throughput and remote operations at large-scale user facilities

Imaging, scattering, and spectroscopy are fundamental in understanding a...
research
10/12/2022

Microscopy is All You Need

We pose that microscopy offers an ideal real-world experimental environm...

Please sign up or login with your details

Forgot password? Click here to reset