Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis

03/29/2021
by   Melanie Schellenberg, et al.
36

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties such as blood oxygenation with high spatial resolution and in an interventional setting. However, decades of research invested in solving the inverse problem of recovering clinically relevant tissue properties from spectral measurements have failed to produce solutions that can quantify tissue parameters robustly in a clinical setting. Previous attempts to address the limitations of model-based approaches with machine learning were hampered by the absence of labeled reference data needed for supervised algorithm training. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains a huge unsolved challenge. As a first step to address this bottleneck, we propose a novel approach to PAT data simulation, which we refer to as "learning to simulate". Our approach involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, represented by semantic segmentation maps and (2) pixel-wise assignment of corresponding optical and acoustic properties. In the present work, we focus on the first challenge. Specifically, we leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries. According to an initial in silico feasibility study our approach is well-suited for contributing to realistic PAT image synthesis and could thus become a fundamental step for deep learning-based quantitative PAT.

READ FULL TEXT

page 2

page 4

page 6

page 10

research
08/11/2020

SAFRON: Stitching Across the Frontier for Generating Colorectal Cancer Histology Images

Synthetic images can be used for the development and evaluation of deep ...
research
06/11/2023

Moving beyond simulation: data-driven quantitative photoacoustic imaging using tissue-mimicking phantoms

Accurate measurement of optical absorption coefficients from photoacoust...
research
02/15/2019

Estimation of blood oxygenation with learned spectral decoloring for quantitative photoacoustic imaging (LSD-qPAI)

One of the main applications of photoacoustic (PA) imaging is the recove...
research
08/14/2018

ScarGAN: Chained Generative Adversarial Networks to Simulate Pathological Tissue on Cardiovascular MR Scans

Medical images with specific pathologies are scarce, but a large amount ...
research
03/08/2019

Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

Purpose: Optical imaging is evolving as a key technique for advanced sen...
research
11/05/2019

Out of distribution detection for intra-operative functional imaging

Multispectral optical imaging is becoming a key tool in the operating ro...
research
05/20/2021

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic imaging has the potential to revolutionise healthcare due ...

Please sign up or login with your details

Forgot password? Click here to reset