DeepAI AI Chat
Log In Sign Up

Diagnose Like a Radiologist: Hybrid Neuro-Probabilistic Reasoning for Attribute-Based Medical Image Diagnosis

by   Gangming Zhao, et al.
Association for Computing Machinery

During clinical practice, radiologists often use attributes, e.g. morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning and a graph convolutional network branch performing more generic relational modeling and reasoning using a feature representation. Tight coupling between these two branches is achieved via a cross-network attention mechanism and the fusion of their classification results. We have successfully applied our hybrid reasoning algorithm to two challenging medical image diagnosis tasks. On the LIDC-IDRI benchmark dataset for benign-malignant classification of pulmonary nodules in CT images, our method achieves a new state-of-the-art accuracy of 95.36% and an AUC of 96.54%. Our method also achieves a 3.24% accuracy improvement on an in-house chest X-ray image dataset for tuberculosis diagnosis. Our ablation study indicates that our hybrid algorithm achieves a much better generalization performance than a pure neural network architecture under very limited training data.


page 2

page 10

page 11

page 12

page 13


Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets

Chest radiography is the most common medical image examination for scree...

Cross Chest Graph for Disease Diagnosis with Structural Relational Reasoning

Locating lesions is important in the computer-aided diagnosis of X-ray i...

Multi-Instance Learning by Utilizing Structural Relationship among Instances

Multi-Instance Learning(MIL) aims to learn the mapping between a bag of ...

Multi-Scale Feature Fusion using Parallel-Attention Block for COVID-19 Chest X-ray Diagnosis

Under the global COVID-19 crisis, accurate diagnosis of COVID-19 from Ch...

Medical idioms for clinical Bayesian network development

Bayesian Networks (BNs) are graphical probabilistic models that have pro...

OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays

Chest X-ray (CXR) is the most typical medical image worldwide to examine...

Justifying Diagnosis Decisions by Deep Neural Networks

An integrated approach is proposed across visual and textual data to bot...