Contrastive Attention for Automatic Chest X-ray Report Generation

06/13/2021
by   Fenglin Liu, et al.
16

Recently, chest X-ray report generation, which aims to automatically generate descriptions of given chest X-ray images, has received growing research interests. The key challenge of chest X-ray report generation is to accurately capture and describe the abnormal regions. In most cases, the normal regions dominate the entire chest X-ray image, and the corresponding descriptions of these normal regions dominate the final report. Due to such data bias, learning-based models may fail to attend to abnormal regions. In this work, to effectively capture and describe abnormal regions, we propose the Contrastive Attention (CA) model. Instead of solely focusing on the current input image, the CA model compares the current input image with normal images to distill the contrastive information. The acquired contrastive information can better represent the visual features of abnormal regions. According to the experiments on the public IU-X-ray and MIMIC-CXR datasets, incorporating our CA into several existing models can boost their performance across most metrics. In addition, according to the analysis, the CA model can help existing models better attend to the abnormal regions and provide more accurate descriptions which are crucial for an interpretable diagnosis. Specifically, we achieve the state-of-the-art results on the two public datasets.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 9

08/06/2019

Addressing Data Bias Problems for Chest X-ray Image Report Generation

Automatic medical report generation from chest X-ray images is one possi...
03/05/2019

Abnormal Chest X-ray Identification With Generative Adversarial One-Class Classifier

Being one of the most common diagnostic imaging tests, chest radiography...
05/20/2021

AnaXNet: Anatomy Aware Multi-label Finding Classification in Chest X-ray

Radiologists usually observe anatomical regions of chest X-ray images as...
09/25/2021

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

Radiology report generation aims at generating descriptive text from rad...
04/11/2021

Cross-Modal Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop

Building a highly accurate predictive model for these tasks usually requ...
07/30/2019

Exploring large scale public medical image datasets

Rationale and Objectives: Medical artificial intelligence systems are de...

Code Repositories

multi_modal_learning

A literature repo for multi-modal machine learning


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.