VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers

04/14/2021
by   Sharif Amit Kamran, et al.
0

In Fluorescein Angiography (FA), an exogenous dye is injected in the bloodstream to image the vascular structure of the retina. The injected dye can cause adverse reactions such as nausea, vomiting, anaphylactic shock, and even death. In contrast, color fundus imaging is a non-invasive technique used for photographing the retina but does not have sufficient fidelity for capturing its vascular structure. The only non-invasive method for capturing retinal vasculature is optical coherence tomography-angiography (OCTA). However, OCTA equipment is quite expensive, and stable imaging is limited to small areas on the retina. In this paper, we propose a novel conditional generative adversarial network (GAN) capable of simultaneously synthesizing FA images from fundus photographs while predicting retinal degeneration. The proposed system has the benefit of addressing the problem of imaging retinal vasculature in a non-invasive manner as well as predicting the existence of retinal abnormalities. We use a semi-supervised approach to train our GAN using multiple weighted losses on different modalities of data. Our experiments validate that the proposed architecture exceeds recent state-of-the-art generative networks for fundus-to-angiography synthesis. Moreover, our vision transformer-based discriminators generalize quite well on out-of-distribution data sets for retinal disease prediction.

READ FULL TEXT

page 3

page 6

research
05/11/2020

Fundus2Angio: A Novel Conditional GAN Architecture for Generating Fluorescein Angiography Images from Retinal Fundus Photography

Carrying out clinical diagnosis of retinal vascular degeneration using F...
research
06/28/2022

GAN-based Super-Resolution and Segmentation of Retinal Layers in Optical coherence tomography Scans

In this paper, we design a Generative Adversarial Network (GAN)-based so...
research
07/17/2020

Attention2AngioGAN: Synthesizing Fluorescein Angiography from Retinal Fundus Images using Generative Adversarial Networks

Fluorescein Angiography (FA) is a technique that employs the designated ...
research
11/28/2019

Sparse-GAN: Sparsity-constrained Generative Adversarial Network for Anomaly Detection in Retinal OCT Image

With the development of convolutional neural network, deep learning has ...
research
08/01/2023

Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Images

Optical Coherence Tomography (OCT) is a novel and effective screening to...
research
10/17/2017

A New Coherence-Penalized Minimal Path Model with Application to Retinal Vessel Centerline Delineation

In this paper, we propose a new minimal path model for minimally interac...

Please sign up or login with your details

Forgot password? Click here to reset