VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels

by   Saahil Jain, et al.

Automatic extraction of medical conditions from free-text radiology reports is critical for supervising computer vision models to interpret medical images. In this work, we show that radiologists labeling reports significantly disagree with radiologists labeling corresponding chest X-ray images, which reduces the quality of report labels as proxies for image labels. We develop and evaluate methods to produce labels from radiology reports that have better agreement with radiologists labeling images. Our best performing method, called VisualCheXbert, uses a biomedically-pretrained BERT model to directly map from a radiology report to the image labels, with a supervisory signal determined by a computer vision model trained to detect medical conditions from chest X-ray images. We find that VisualCheXbert outperforms an approach using an existing radiology report labeler by an average F1 score of 0.14 (95 also find that VisualCheXbert better agrees with radiologists labeling chest X-ray images than do radiologists labeling the corresponding radiology reports by an average F1 score across several medical conditions of between 0.12 (95 CI 0.09, 0.15) and 0.21 (95


page 1

page 2

page 3

page 4


Effect of Radiology Report Labeler Quality on Deep Learning Models for Chest X-Ray Interpretation

Although deep learning models for chest X-ray interpretation are commonl...

CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT

The extraction of labels from radiology text reports enables large-scale...

Caveats in Generating Medical Imaging Labels from Radiology Reports

Acquiring high-quality annotations in medical imaging is usually a costl...

Interpretation of Mammogram and Chest X-Ray Reports Using Deep Neural Networks - Preliminary Results

Radiology reports are an important means of communication between radiol...

Efficient and Accurate Abnormality Mining from Radiology Reports with Customized False Positive Reduction

Obtaining datasets labeled to facilitate model development is a challeng...

Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training

When reading images, radiologists generate text reports describing the f...

PneumoXttention: A CNN compensating for Human Fallibility when Detecting Pneumonia through CXR images with Attention

Automatic Chest Radiograph X-ray (CXR) interpretation by machines is an ...

Code Repositories


Addressing the Discrepancy Between Radiology Report Labels and Image Labels

view repo