3D-POP – An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

by   Hemal Naik, et al.

Recent advances in machine learning and computer vision are revolutionizing the field of animal behavior by enabling researchers to track the poses and locations of freely moving animals without any marker attachment. However, large datasets of annotated images of animals for markerless pose tracking, especially high-resolution images taken from multiple angles with accurate 3D annotations, are still scant. Here, we propose a method that uses a motion capture (mo-cap) system to obtain a large amount of annotated data on animal movement and posture (2D and 3D) in a semi-automatic manner. Our method is novel in that it extracts the 3D positions of morphological keypoints (e.g eyes, beak, tail) in reference to the positions of markers attached to the animals. Using this method, we obtained, and offer here, a new dataset - 3D-POP with approximately 300k annotated frames (4 million instances) in the form of videos having groups of one to ten freely moving birds from 4 different camera views in a 3.6m x 4.2m area. 3D-POP is the first dataset of flocking birds with accurate keypoint annotations in 2D and 3D along with bounding box and individual identities and will facilitate the development of solutions for problems of 2D to 3D markerless pose, trajectory tracking, and identification in birds.


page 5

page 7

page 8


Motion Capture from Pan-Tilt Cameras with Unknown Orientation

In sports, such as alpine skiing, coaches would like to know the speed a...

Automatically tracking neurons in a moving and deforming brain

Advances in optical neuroimaging techniques now allow neural activity to...

Amur Tiger Re-identification in the Wild

Monitoring the population and movements of endangered species is an impo...

Evaluation of deep lift pose models for 3D rodent pose estimation based on geometrically triangulated data

The assessment of laboratory animal behavior is of central interest in m...

Video Annotation for Visual Tracking via Selection and Refinement

Deep learning based visual trackers entail offline pre-training on large...

TaiChi Action Capture and Performance Analysis with Multi-view RGB Cameras

Recent advances in computer vision and deep learning have influenced the...

3D BAT: A Semi-Automatic, Web-based 3D Annotation Toolbox for Full-Surround, Multi-Modal Data Streams

In this paper, we focus on obtaining 2D and 3D labels, as well as track ...

Please sign up or login with your details

Forgot password? Click here to reset