RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation

by   Shiqi Huang, et al.

Automatic diabetic retinopathy (DR) lesions segmentation makes great sense of assisting ophthalmologists in diagnosis. Although many researches have been conducted on this task, most prior works paid too much attention to the designs of networks instead of considering the pathological association for lesions. Through investigating the pathogenic causes of DR lesions in advance, we found that certain lesions are closed to specific vessels and present relative patterns to each other. Motivated by the observation, we propose a relation transformer block (RTB) to incorporate attention mechanisms at two main levels: a self-attention transformer exploits global dependencies among lesion features, while a cross-attention transformer allows interactions between lesion and vessel features by integrating valuable vascular information to alleviate ambiguity in lesion detection caused by complex fundus structures. In addition, to capture the small lesion patterns first, we propose a global transformer block (GTB) which preserves detailed information in deep network. By integrating the above blocks of dual-branches, our network segments the four kinds of lesions simultaneously. Comprehensive experiments on IDRiD and DDR datasets well demonstrate the superiority of our approach, which achieves competitive performance compared to state-of-the-arts.


page 1

page 5

page 8

page 9

page 10


Learn to Segment Retinal Lesions and Beyond

Towards automated retinal screening, this paper makes an endeavor to sim...

Meta-information-aware Dual-path Transformer for Differential Diagnosis of Multi-type Pancreatic Lesions in Multi-phase CT

Pancreatic cancer is one of the leading causes of cancer-related death. ...

Lesion Segmentation and RECIST Diameter Prediction via Click-driven Attention and Dual-path Connection

Measuring lesion size is an important step to assess tumor growth and mo...

Multi-dimension unified Swin Transformer for 3D Lesion Segmentation in Multiple Anatomical Locations

In oncology research, accurate 3D segmentation of lesions from CT scans ...

View-Disentangled Transformer for Brain Lesion Detection

Deep neural networks (DNNs) have been widely adopted in brain lesion det...

Transformer Lesion Tracker

Evaluating lesion progression and treatment response via longitudinal le...

Please sign up or login with your details

Forgot password? Click here to reset