EgoTransfer: Transferring Motion Across Egocentric and Exocentric Domains using Deep Neural Networks

12/17/2016
by   Shervin Ardeshir, et al.
0

Mirror neurons have been observed in the primary motor cortex of primate species, in particular in humans and monkeys. A mirror neuron fires when a person performs a certain action, and also when he observes the same action being performed by another person. A crucial step towards building fully autonomous intelligent systems with human-like learning abilities is the capability in modeling the mirror neuron. On one hand, the abundance of egocentric cameras in the past few years has offered the opportunity to study a lot of vision problems from the first-person perspective. A great deal of interesting research has been done during the past few years, trying to explore various computer vision tasks from the perspective of the self. On the other hand, videos recorded by traditional static cameras, capture humans performing different actions from an exocentric third-person perspective. In this work, we take the first step towards relating motion information across these two perspectives. We train models that predict motion in an egocentric view, by observing it from an exocentric view, and vice versa. This allows models to predict how an egocentric motion would look like from outside. To do so, we train linear and nonlinear models and evaluate their performance in terms of retrieving the egocentric (exocentric) motion features, while having access to an exocentric (egocentric) motion feature. Our experimental results demonstrate that motion information can be successfully transferred across the two views.

READ FULL TEXT

page 1

page 4

page 6

research
08/13/2019

Predicting 3D Human Dynamics from Video

Given a video of a person in action, we can easily guess the 3D future m...
research
06/15/2016

Ego-Surfing: Person Localization in First-Person Videos Using Ego-Motion Signatures

We envision a future time when wearable cameras are worn by the masses a...
research
06/20/2014

Predicting Motivations of Actions by Leveraging Text

Understanding human actions is a key problem in computer vision. However...
research
05/18/2021

Multi-Person Extreme Motion Prediction with Cross-Interaction Attention

Human motion prediction aims to forecast future human poses given a sequ...
research
12/23/2019

Analysis of the hands in egocentric vision: A survey

Egocentric vision (a.k.a. first-person vision - FPV) applications have t...
research
12/24/2016

EgoReID: Cross-view Self-Identification and Human Re-identification in Egocentric and Surveillance Videos

Human identification remains to be one of the challenging tasks in compu...
research
06/01/2022

Binding Dancers Into Attractors

To effectively perceive and process observations in our environment, fea...

Please sign up or login with your details

Forgot password? Click here to reset