Unsupervised Shot Boundary Detection for Temporal Segmentation of Long Capsule Endoscopy Videos

10/18/2021
by   Sodiq Adewole, et al.
0

Physicians use Capsule Endoscopy (CE) as a non-invasive and non-surgical procedure to examine the entire gastrointestinal (GI) tract for diseases and abnormalities. A single CE examination could last between 8 to 11 hours generating up to 80,000 frames which is compiled as a video. Physicians have to review and analyze the entire video to identify abnormalities or diseases before making diagnosis. This review task can be very tedious, time consuming and prone to error. While only as little as a single frame may capture useful content that is relevant to the physicians' final diagnosis, frames covering the small bowel region alone could be as much as 50,000. To minimize physicians' review time and effort, this paper proposes a novel unsupervised and computationally efficient temporal segmentation method to automatically partition long CE videos into a homogeneous and identifiable video segments. However, the search for temporal boundaries in a long video using high dimensional frame-feature matrix is computationally prohibitive and impracticable for real clinical application. Therefore, leveraging both spatial and temporal information in the video, we first extracted high level frame features using a pretrained CNN model and then projected the high-dimensional frame-feature matrix to lower 1-dimensional embedding. Using this 1-dimensional sequence embedding, we applied the Pruned Exact Linear Time (PELT) algorithm to searched for temporal boundaries that indicates the transition points from normal to abnormal frames and vice-versa. We experimented with multiple real patients' CE videos and our model achieved an AUC of 66% on multiple test videos against expert provided labels.

READ FULL TEXT

page 1

page 3

page 4

page 7

research
10/18/2021

Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos

Temporal activity localization in long videos is an important problem. T...
research
04/07/2021

Deep Transformers for Fast Small Intestine Grounding in Capsule Endoscope Video

Capsule endoscopy is an evolutional technique for examining and diagnosi...
research
05/08/2013

Automated polyp detection in colon capsule endoscopy

Colorectal polyps are important precursors to colon cancer, a major heal...
research
01/24/2023

Detecting and measuring human gastric peristalsis using magnetically controlled capsule endoscope

Magnetically controlled capsule endoscope (MCCE) is an emerging tool for...
research
01/11/2021

Lesion2Vec: Deep Metric Learning for Few-Shot Multiple Lesions Recognition in Wireless Capsule Endoscopy Video

Effective and rapid detection of lesions in the Gastrointestinal tract i...
research
03/09/2023

TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering

Temporal action segmentation in untrimmed videos has gained increased at...

Please sign up or login with your details

Forgot password? Click here to reset