Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organs

by   Guankun Wang, et al.

Video-assisted transoral tracheal intubation (TI) necessitates using an endoscope that helps the physician insert a tracheal tube into the glottis instead of the esophagus. The growing trend of robotic-assisted TI would require a medical robot to distinguish anatomical features like an experienced physician which can be imitated by utilizing supervised deep-learning techniques. However, the real datasets of oropharyngeal organs are often inaccessible due to limited open-source data and patient privacy. In this work, we propose a domain adaptive Sim-to-Real framework called IoU-Ranking Blend-ArtFlow (IRB-AF) for image segmentation of oropharyngeal organs. The framework includes an image blending strategy called IoU-Ranking Blend (IRB) and style-transfer method ArtFlow. Here, IRB alleviates the problem of poor segmentation performance caused by significant datasets domain differences; while ArtFlow is introduced to reduce the discrepancies between datasets further. A virtual oropharynx image dataset generated by the SOFA framework is used as the learning subject for semantic segmentation to deal with the limited availability of actual endoscopic images. We adapted IRB-AF with the state-of-the-art domain adaptive segmentation models. The results demonstrate the superior performance of our approach in further improving the segmentation accuracy and training stability.


page 3

page 6

page 8

page 10

page 14


Sim-to-Real Segmentation in Robot-assisted Transoral Tracheal Intubation

Robotic-assisted tracheal intubation requires the robot to distinguish a...

Dual Adaptive Pyramid Network for Cross-Stain Histopathology Image Segmentation

Supervised semantic segmentation normally assumes the test data being in...

Random Style Transfer based Domain Generalization Networks Integrating Shape and Spatial Information

Deep learning (DL)-based models have demonstrated good performance in me...

Cross-View Regularization for Domain Adaptive Panoptic Segmentation

Panoptic segmentation unifies semantic segmentation and instance segment...

Synthetic-to-real Composite Semantic Segmentation in Additive Manufacturing

The application of computer vision and machine learning methods in the f...

Please sign up or login with your details

Forgot password? Click here to reset