Chest ImaGenome Dataset for Clinical Reasoning

07/31/2021
by   Joy T. Wu, et al.
0

Despite the progress in automatic detection of radiologic findings from chest X-ray (CXR) images in recent years, a quantitative evaluation of the explainability of these models is hampered by the lack of locally labeled datasets for different findings. With the exception of a few expert-labeled small-scale datasets for specific findings, such as pneumonia and pneumothorax, most of the CXR deep learning models to date are trained on global "weak" labels extracted from text reports, or trained via a joint image and unstructured text learning strategy. Inspired by the Visual Genome effort in the computer vision community, we constructed the first Chest ImaGenome dataset with a scene graph data structure to describe 242,072 images. Local annotations are automatically produced using a joint rule-based natural language processing (NLP) and atlas-based bounding box detection pipeline. Through a radiologist constructed CXR ontology, the annotations for each CXR are connected as an anatomy-centered scene graph, useful for image-level reasoning and multimodal fusion applications. Overall, we provide: i) 1,256 combinations of relation annotations between 29 CXR anatomical locations (objects with bounding box coordinates) and their attributes, structured as a scene graph per image, ii) over 670,000 localized comparison relations (for improved, worsened, or no change) between the anatomical locations across sequential exams, as well as ii) a manually annotated gold standard scene graph dataset from 500 unique patients.

READ FULL TEXT

page 7

page 23

page 24

research
01/23/2020

Localization of Critical Findings in Chest X-Ray without Local Annotations Using Multi-Instance Learning

The automatic detection of critical findings in chest X-rays (CXR), such...
research
08/30/2023

Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting

The task of radiology reporting comprises describing and interpreting th...
research
03/29/2022

Few-shot Structured Radiology Report Generation Using Natural Language Prompts

Chest radiograph reporting is time-consuming, and numerous solutions to ...
research
03/20/2022

VinDr-PCXR: An open, large-scale chest radiograph dataset for interpretation of common thoracic diseases in children

Computer-aided diagnosis systems in adult chest radiography (CXR) have r...
research
02/07/2018

Generating Triples with Adversarial Networks for Scene Graph Construction

Driven by successes in deep learning, computer vision research has begun...
research
07/31/2020

Weakly supervised one-stage vision and language disease detection using large scale pneumonia and pneumothorax studies

Detecting clinically relevant objects in medical images is a challenge d...
research
01/03/2020

Discoverability in Satellite Imagery: A Good Sentence is Worth a Thousand Pictures

Small satellite constellations provide daily global coverage of the eart...

Please sign up or login with your details

Forgot password? Click here to reset