A Robotic 3D Perception System for Operating Room Environment Awareness

03/20/2020
by   Zhaoshuo Li, et al.
0

Purpose: We describe a 3D multi-view perception system for the da Vinci surgical system to enable Operating room (OR) scene understanding and context awareness. Methods: Our proposed system is comprised of four Time-of-Flight (ToF) cameras rigidly attached to strategic locations on the daVinci Xi patient side cart (PSC). The cameras are registered to the robot's kinematic chain by performing a one-time calibration routine and therefore, information from all cameras can be fused and represented in one common coordinate frame. Based on this architecture, a multi-view 3D scene semantic segmentation algorithm is created to enable recognition of common and salient objects/equipment and surgical activities in a da Vinci OR. Our proposed 3D semantic segmentation method has been trained and validated on a novel densely annotated dataset that has been captured from clinical scenarios. Results: The results show that our proposed architecture has acceptable registration error (3.3%±1.4% of object-camera distance) and can robustly improve scene segmentation performance (mean Intersection Over Union - mIOU) for less frequently appearing classes (> 0.013) compared to a single-view method. Conclusion: We present the first dynamic multi-view perception system with a novel segmentation architecture, which can be used as a building block technology for applications such as surgical workflow analysis, automation of surgical sub-tasks and advanced guidance systems.

READ FULL TEXT

page 8

page 11

page 19

page 27

research
04/12/2022

RGB-D Semantic SLAM for Surgical Robot Navigation in the Operating Room

Gaining spatial awareness of the Operating Room (OR) for surgical roboti...
research
07/26/2023

DisguisOR: Holistic Face Anonymization for the Operating Room

Purpose: Recent advances in Surgical Data Science (SDS) have contributed...
research
05/30/2022

MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic Segmentation

We present MVMO (Multi-View, Multi-Object dataset): a synthetic dataset ...
research
06/29/2020

Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery

Automatic recognition of surgical activities in the operating room (OR) ...
research
02/06/2021

A surgical dataset from the da Vinci Research Kit for task automation and recognition

The use of datasets is getting more relevance in surgical robotics since...
research
03/16/2022

Know your sensORs x2013 A Modality Study For Surgical Action Classification

The surgical operating room (OR) presents many opportunities for automat...
research
08/03/2022

AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomy

Computer-assisted minimally invasive surgery has great potential in bene...

Please sign up or login with your details

Forgot password? Click here to reset