RadTex: Learning Efficient Radiograph Representations from Text Reports

08/05/2022
by   Keegan Quigley, et al.
0

Automated analysis of chest radiography using deep learning has tremendous potential to enhance the clinical diagnosis of diseases in patients. However, deep learning models typically require large amounts of annotated data to achieve high performance – often an obstacle to medical domain adaptation. In this paper, we build a data-efficient learning framework that utilizes radiology reports to improve medical image classification performance with limited labeled data (fewer than 1000 examples). Specifically, we examine image-captioning pretraining to learn high-quality medical image representations that train on fewer examples. Following joint pretraining of a convolutional encoder and transformer decoder, we transfer the learned encoder to various classification tasks. Averaged over 9 pathologies, we find that our model achieves higher classification performance than ImageNet-supervised and in-domain supervised pretraining when labeled training data is limited.

READ FULL TEXT
research
01/25/2023

Self-Supervised Curricular Deep Learning for Chest X-Ray Image Classification

Deep learning technologies have already demonstrated a high potential to...
research
06/11/2020

VirTex: Learning Visual Representations from Textual Annotations

The de-facto approach to many vision tasks is to start from pretrained v...
research
05/04/2023

Forward-Forward Contrastive Learning

Medical image classification is one of the most important tasks for comp...
research
10/02/2020

Contrastive Learning of Medical Visual Representations from Paired Images and Text

Learning visual representations of medical images is core to medical ima...
research
10/06/2020

Weakly-Supervised Feature Learning via Text and Image Matching

When training deep neural networks for medical image classification, obt...
research
05/21/2021

Visual representation of negation: Real world data analysis on comic image design

There has been a widely held view that visual representations (e.g., pho...
research
05/05/2022

Understanding Transfer Learning for Chest Radiograph Clinical Report Generation with Modified Transformer Architectures

The image captioning task is increasingly prevalent in artificial intell...

Please sign up or login with your details

Forgot password? Click here to reset