Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy

by   Sergei V. Kalinin, et al.

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment (AE) in imaging. Here, we aim to analyze the major pathways towards AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment, and consider the latencies, biases, and knowledge priors of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning.



There are no comments yet.


page 5

page 8

page 12

page 17

page 19

page 24

page 31

page 34


An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Artificial intelligence (AI) promises to reshape scientific inquiry and ...

Embedding Human Heuristics in Machine-Learning-Enabled Probe Microscopy

Scanning probe microscopists generally do not rely on complete images to...

Reduced Electron Exposure for Energy-Dispersive Spectroscopy using Dynamic Sampling

Analytical electron microscopy and spectroscopy of biological specimens,...

Scanning Probe State Recognition With Multi-Class Neural Network Ensembles

One of the largest obstacles facing scanning probe microscopy is the con...

Problem-fluent models for complex decision-making in autonomous materials research

We review our recent work in the area of autonomous materials research, ...

Histogram-based Auto Segmentation: A Novel Approach to Segmenting Integrated Circuit Structures from SEM Images

In the Reverse Engineering and Hardware Assurance domain, a majority of ...

Machine learning for faster and smarter fluorescence lifetime imaging microscopy

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.