You-Do, I-Learn: Unsupervised Multi-User egocentric Approach Towards Video-Based Guidance

10/16/2015
by   Dima Damen, et al.
0

This paper presents an unsupervised approach towards automatically extracting video-based guidance on object usage, from egocentric video and wearable gaze tracking, collected from multiple users while performing tasks. The approach i) discovers task relevant objects, ii) builds a model for each, iii) distinguishes different ways in which each discovered object has been used and iv) discovers the dependencies between object interactions. The work investigates using appearance, position, motion and attention, and presents results using each and a combination of relevant features. Moreover, an online scalable approach is presented and is compared to offline results. The paper proposes a method for selecting a suitable video guide to be displayed to a novice user indicating how to use an object, purely triggered by the user's gaze. The potential assistive mode can also recommend an object to be used next based on the learnt sequence of object interactions. The approach was tested on a variety of daily tasks such as initialising a printer, preparing a coffee and setting up a gym machine.

READ FULL TEXT

page 2

page 9

page 19

page 22

page 23

page 26

page 28

page 29

research
07/14/2018

GaGARoS: A Gaze Guided Assistive Robotic System for Daily-Living Activities

Patients suffering from tetraplegia or quadriplegia have limited body mo...
research
07/14/2018

Free-View, 3D Gaze-Guided, Assistive Robotic System for Activities of Daily Living

Patients suffering from quadriplegia have limited body motion which prev...
research
10/15/2018

I Can See Your Aim: Estimating User Attention From Gaze For Handheld Robot Collaboration

This paper explores the estimation of user attention in the setting of a...
research
11/10/2022

Efficient Unsupervised Video Object Segmentation Network Based on Motion Guidance

Considerable unsupervised video object segmentation algorithms based on ...
research
06/12/2020

Multiple-Vehicle Tracking in the Highway Using Appearance Model and Visual Object Tracking

In recent decades, due to the groundbreaking improvements in machine vis...
research
11/19/2021

Xp-GAN: Unsupervised Multi-object Controllable Video Generation

Video Generation is a relatively new and yet popular subject in machine ...
research
03/06/2020

Toward Adaptive Guidance: Modeling the Variety of User Behaviors in Continuous-Skill-Improving Experiences of Machine Operation Tasks

An adaptive guidance system that supports equipment operators requires a...

Please sign up or login with your details

Forgot password? Click here to reset