Gaze-Vergence-Controlled See-Through Vision in Augmented Reality

by   Zhimin Wang, et al.

Augmented Reality (AR) see-through vision is an interesting research topic since it enables users to see through a wall and see the occluded objects. Most existing research focuses on the visual effects of see-through vision, while the interaction method is less studied. However, we argue that using common interaction modalities, e.g., midair click and speech, may not be the optimal way to control see-through vision. This is because when we want to see through something, it is physically related to our gaze depth/vergence and thus should be naturally controlled by the eyes. Following this idea, this paper proposes a novel gaze-vergence-controlled (GVC) see-through vision technique in AR. Since gaze depth is needed, we build a gaze tracking module with two infrared cameras and the corresponding algorithm and assemble it into the Microsoft HoloLens 2 to achieve gaze depth estimation. We then propose two different GVC modes for see-through vision to fit different scenarios. Extensive experimental results demonstrate that our gaze depth estimation is efficient and accurate. By comparing with conventional interaction modalities, our GVC techniques are also shown to be superior in terms of efficiency and more preferred by users. Finally, we present four example applications of gaze-vergence-controlled see-through vision.


The Markup Language for Designing Gaze Controlled Applications

The Gaze Interaction Markup Language (GIML) is presented, which is new l...

Interaction Design of Dwell Selection Toward Gaze-based AR/VR Interaction

In this paper, we first position the current dwell selection among gaze-...

Automatic Gaze Analysis: A Survey of DeepLearning based Approaches

Eye gaze analysis is an important research problem in the field of compu...

Dynamic X-Ray Vision in Mixed Reality

X-ray vision, a technique that allows users to see through walls and oth...

Back to the Future: Revisiting Mouse and Keyboard Interaction for HMD-based Immersive Analytics

With the rise of natural user interfaces, immersive analytics applicatio...

Foveated Haptic Gaze

As digital worlds become ubiquitous via video games, simulations, virtua...

Model-based Task Analysis and Large-scale Video-based Remote Evaluation Methods for Extended Reality Research

In this paper, we introduce two remote extended reality (XR) research me...