SHD360: A Benchmark Dataset for Salient Human Detection in 360° Videos

05/24/2021
by   Yi Zhang, et al.
7

Salient human detection (SHD) in dynamic 360 immersive videos is of great importance for various applications such as robotics, inter-human and human-object interaction in augmented reality. However, 360 video SHD has been seldom discussed in the computer vision community due to a lack of datasets with large-scale omnidirectional videos and rich annotations. To this end, we propose SHD360, the first 360 video SHD dataset collecting various real-life daily scenes, providing six-level hierarchical annotations for 6,268 key frames uniformly sampled from 37,403 omnidirectional video frames at 4K resolution. Specifically, each collected key frame is labeled with a super-class, a sub-class, associated attributes (e.g., geometrical distortion), bounding boxes and per-pixel object-/instance-level masks. As a result, our SHD360 contains totally 16,238 salient human instances with manually annotated pixel-wise ground truth. Since so far there is no method proposed for 360 SHD, we systematically benchmark 11 representative state-of-the-art salient object detection (SOD) approaches on our SHD360, and explore key issues derived from extensive experimenting results. We hope our proposed dataset and benchmark could serve as a good starting point for advancing human-centric researches towards 360 panoramic data. Our dataset and benchmark will be publicly available at https://github.com/PanoAsh/SHD360.

READ FULL TEXT

page 2

page 5

page 6

page 7

07/24/2021

ASOD60K: Audio-Induced Salient Object Detection in Panoramic Videos

Exploring to what humans pay attention in dynamic panoramic scenes is us...
03/10/2020

PANDA: A Gigapixel-level Human-centric Video Dataset

We present PANDA, the first gigaPixel-level humAN-centric viDeo dAtaset,...
01/11/2021

Horizontal-to-Vertical Video Conversion

Alongside the prevalence of mobile videos, the general public leans towa...
01/22/2020

A Fixation-based 360° Benchmark Dataset for Salient Object Detection

Fixation prediction (FP) in panoramic contents has been widely investiga...
12/13/2020

FSOCO: The Formula Student Objects in Context Dataset

This paper presents the FSOCO dataset, a collaborative dataset for visio...
12/18/2020

Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations

3D object detection has recently become popular due to many applications...
09/27/2022

View-aware Salient Object Detection for 360° Omnidirectional Image

Image-based salient object detection (ISOD) in 360 scenarios is signific...

Code Repositories