DeepAI AI Chat
Log In Sign Up

Microscopy is All You Need

by   Sergei V. Kalinin, et al.

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALLE and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogeneous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to APIs facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to creating a novel set of development targets for the ML community by accelerating both real-world ML applications and scientific progress.


page 5

page 7

page 8

page 13

page 15

page 17

page 25


Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods, and the emerging availabili...

Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becomin...

Machine learning for faster and smarter fluorescence lifetime imaging microscopy

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique ...

Machine learning for automated quality control in injection moulding manufacturing

Machine learning (ML) may improve and automate quality control (QC) in i...

Using CycleGANs to Generate Realistic STEM Images for Machine Learning

The rise of automation and machine learning (ML) in electron microscopy ...

SELFIES: a robust representation of semantically constrained graphs with an example application in chemistry

Graphs are ideal representations of complex, relational information. The...