DeepAI AI Chat
Log In Sign Up

MemX: An Attention-Aware Smart Eyewear System for Personalized Moment Auto-capture

05/03/2021
by   Yuhu Chang, et al.
University of Michigan
University of Colorado Boulder
Imperial College London
FUDAN University
10

This work presents MemX: a biologically-inspired attention-aware eyewear system developed with the goal of pursuing the long-awaited vision of a personalized visual Memex. MemX captures human visual attention on the fly, analyzes the salient visual content, and records moments of personal interest in the form of compact video snippets. Accurate attentive scene detection and analysis on resource-constrained platforms is challenging because these tasks are computation and energy intensive. We propose a new temporal visual attention network that unifies human visual attention tracking and salient visual content analysis. Attention tracking focuses computation-intensive video analysis on salient regions, while video analysis makes human attention detection and tracking more accurate. Using the YouTube-VIS dataset and 30 participants, we experimentally show that MemX significantly improves the attention tracking accuracy over the eye-tracking-alone method, while maintaining high system energy efficiency. We have also conducted 11 in-field pilot studies across a range of daily usage scenarios, which demonstrate the feasibility and potential benefits of MemX.

READ FULL TEXT

page 3

page 12

page 13

page 16

page 18

page 20

02/27/2020

A Proto-Object Based Dynamic Visual Saliency Model with an FPGA Implementation

The ability to attend to salient regions of a visual scene is an innate ...
02/27/2020

A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model with an FPGA Implementation

The ability to attend to salient regions of a visual scene is an innate ...
10/27/2022

BI AVAN: Brain inspired Adversarial Visual Attention Network

Visual attention is a fundamental mechanism in the human brain, and it i...
01/01/2019

Not All Words are Equal: Video-specific Information Loss for Video Captioning

An ideal description for a given video should fix its gaze on salient an...
07/22/2013

Is Bottom-Up Attention Useful for Scene Recognition?

The human visual system employs a selective attention mechanism to under...
10/09/2018

Deep Attentive Tracking via Reciprocative Learning

Visual attention, derived from cognitive neuroscience, facilitates human...