Forecasting User Attention During Everyday Mobile Interactions Using Device-Integrated and Wearable Sensors

by   Julian Steil, et al.

Users' visual attention is highly fragmented during mobile interactions but the erratic nature of these attention shifts currently limits attentive user interfaces to adapt after the fact, i.e. after shifts have already happened, thereby severely limiting the adaptation capabilities and user experience. To address these limitations, we study attention forecasting -- the challenging task of predicting whether users' overt visual attention (gaze) will shift between a mobile device and environment in the near future or how long users' attention will stay in a given location. To facilitate the development and evaluation of methods for attention forecasting, we present a novel long-term dataset of everyday mobile phone interactions, continuously recorded from 20 participants engaged in common activities on a university campus over 4.5 hours each (more than 90 hours in total). As a first step towards a fully-fledged attention forecasting interface, we further propose a proof-of-concept method that uses device-integrated sensors and body-worn cameras to encode rich information on device usage and users' visual scene. We demonstrate the feasibility of forecasting bidirectional attention shifts between the device and the environment as well as for predicting the first and total attention span on the device and environment using our method. We further study the impact of different sensors and feature sets on performance and discuss the significant potential but also remaining challenges of forecasting user attention during mobile interactions.


page 2

page 9

page 10

page 11

page 14

page 15

page 17

page 18


Accurate and Robust Eye Contact Detection During Everyday Mobile Device Interactions

Quantification of human attention is key to several tasks in mobile huma...

How far are we from quantifying visual attention in mobile HCI?

With an ever-increasing number of mobile devices competing for our atten...

Mobile Device Type Substitution

Mobile users today interact with a variety of mobile device types includ...

Visual Behaviors and Mobile Information Acquisition

It is common for people to engage in information acquisition tasks while...

Securing Interactive Sessions Using Mobile Device through Visual Channel and Visual Inspection

Communication channel established from a display to a device's camera is...

A Model for Using Physiological Conditions for Proactive Tourist Recommendations

Mobile proactive tourist recommender systems can support tourists by rec...

Exploring Biofeedback with a Tangible Interface Designed for Relaxation

Anxiety is a common health issue that can occur throughout one's existen...

Please sign up or login with your details

Forgot password? Click here to reset