Augmented Reality for Depth Cues in Monocular Minimally Invasive Surgery

03/01/2017
by   Long Chen, et al.
0

One of the major challenges in Minimally Invasive Surgery (MIS) such as laparoscopy is the lack of depth perception. In recent years, laparoscopic scene tracking and surface reconstruction has been a focus of investigation to provide rich additional information to aid the surgical process and compensate for the depth perception issue. However, robust 3D surface reconstruction and augmented reality with depth perception on the reconstructed scene are yet to be reported. This paper presents our work in this area. First, we adopt a state-of-the-art visual simultaneous localization and mapping (SLAM) framework - ORB-SLAM - and extend the algorithm for use in MIS scenes for reliable endoscopic camera tracking and salient point mapping. We then develop a robust global 3D surface reconstruction frame- work based on the sparse point clouds extracted from the SLAM framework. Our approach is to combine an outlier removal filter within a Moving Least Squares smoothing algorithm and then employ Poisson surface reconstruction to obtain smooth surfaces from the unstructured sparse point cloud. Our proposed method has been quantitatively evaluated compared with ground-truth camera trajectories and the organ model surface we used to render the synthetic simulation videos. In vivo laparoscopic videos used in the tests have demonstrated the robustness and accuracy of our proposed framework on both camera tracking and surface reconstruction, illustrating the potential of our algorithm for depth augmentation and depth-corrected augmented reality in MIS with monocular endoscopes.

READ FULL TEXT

page 9

page 11

page 13

research
08/29/2016

ORBSLAM-based Endoscope Tracking and 3D Reconstruction

We aim to track the endoscope location inside the surgical scene and pro...
research
08/20/2021

A Multiple-View Geometric Model for Specularity Prediction on Non-Uniformly Curved Surfaces

Specularity prediction is essential to many computer vision applications...
research
06/07/2020

DeepRelativeFusion: Dense Monocular SLAM using Single-Image Relative Depth Prediction

Traditional monocular visual simultaneous localization and mapping (SLAM...
research
05/25/2017

SLAM based Quasi Dense Reconstruction For Minimally Invasive Surgery Scenes

Recovering surgical scene structure in laparoscope surgery is crucial st...
research
06/04/2022

C^3Fusion: Consistent Contrastive Colon Fusion, Towards Deep SLAM in Colonoscopy

3D colon reconstruction from Optical Colonoscopy (OC) to detect non-exam...
research
08/07/2017

A Solution for Crime Scene Reconstruction using Time-of-Flight Cameras

In this work, we propose a method for three-dimensional (3D) reconstruct...
research
05/26/2020

What the HoloLens Maps Is Your Workspace: Fast Mapping and Set-up of Robot Cells via Head Mounted Displays and Augmented Reality

Classical methods of modelling and mapping robot work cells are time con...

Please sign up or login with your details

Forgot password? Click here to reset