Spatio-temporal Vision Transformer for Super-resolution Microscopy

02/28/2022
by   Charles N. Christensen, et al.
11

Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. Reconstruction of SIM data is prone to artefacts, which becomes problematic when imaging highly dynamic samples because previous methods rely on the assumption that samples are static. We propose a new transformer-based reconstruction method, VSR-SIM, that uses shifted 3-dimensional window multi-head attention in addition to channel attention mechanism to tackle the problem of video super-resolution (VSR) in SIM. The attention mechanisms are found to capture motion in sequences without the need for common motion estimation techniques such as optical flow. We take an approach to training the network that relies solely on simulated data using videos of natural scenery with a model for SIM image formation. We demonstrate a use case enabled by VSR-SIM referred to as rolling SIM imaging, which increases temporal resolution in SIM by a factor of 9. Our method can be applied to any SIM setup enabling precise recordings of dynamic processes in biomedical research with high temporal resolution.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 9

page 10

research
06/03/2022

Dynamic Structured Illumination Microscopy with a Neural Space-time Model

Structured illumination microscopy (SIM) reconstructs a super-resolved i...
research
03/20/2022

Optical Flow for Video Super-Resolution: A Survey

Video super-resolution is currently one of the most active research topi...
research
03/24/2020

ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images

Structured illumination microscopy (SIM) has become an important techniq...
research
11/16/2022

SATVSR: Scenario Adaptive Transformer for Cross Scenarios Video Super-Resolution

Video Super-Resolution (VSR) aims to recover sequences of high-resolutio...
research
03/08/2018

Dynamic Spike Super-resolution and Applications to Ultrafast Ultrasound Imaging

We consider the dynamical super-resolution problem consisting in the rec...
research
02/19/2016

Structured illumination microscopy image reconstruction algorithm

Structured illumination microscopy (SIM) is a very important super-resol...
research
11/06/2020

Augmented Equivariant Attention Networks for Electron Microscopy Image Super-Resolution

Taking electron microscopy (EM) images in high-resolution is time-consum...

Please sign up or login with your details

Forgot password? Click here to reset