On the Automatic Generation of Medical Imaging Reports

by   Baoyu Jing, et al.
Petuum, Inc.

Medical imaging is widely used in clinical practice for diagnosis and treatment. Specialized physicians read medical images and write textual reports to narrate the findings. Report-writing can be error-prone for unexperienced physicians, and time-consuming and tedious for physicians in highly populated nations. To address these issues, we study the automatic generation of medical imaging reports, as an assistance for human physicians in producing reports more accurately and efficiently. This task presents several challenges. First, a complete report contains multiple heterogeneous forms of information, including findings which are paragraphs and tags which are a list of key words. Second, abnormal regions in medical images are difficult to identify. Generating textual narrations for them is even harder. Third, the reports are typically long, containing multiple paragraphs. To cope with these challenges, we (1) build a multi-task learning framework which jointly performs the prediction of tags and the generation of paragraphs, (2) propose a co-attention mechanism to localize regions containing abnormalities and generate narrations for them, (3) develop a hierarchical LSTM model to generate long para- graphs. We demonstrate the effectiveness of the proposed methods on a chest x-ray dataset and a pathology dataset.


page 1

page 3

page 7

page 8


Generating Radiology Reports via Memory-driven Transformer

Medical imaging is frequently used in clinical practice and trials for d...

Confidence-Guided Radiology Report Generation

Medical imaging plays a pivotal role in diagnosis and treatment in clini...

Lesion Guided Explainable Few Weak-shot Medical Report Generation

Medical images are widely used in clinical practice for diagnosis. Autom...

Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledge

Among all the sub-sections in a typical radiology report, the Clinical I...

Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning

Medical imaging technologies, including computed tomography (CT) or ches...

XRayGAN: Consistency-preserving Generation of X-ray Images from Radiology Reports

To effectively train medical students to become qualified radiologists, ...

Please sign up or login with your details

Forgot password? Click here to reset