"Hey Model!" – Natural User Interactions and Agency in Accessible Interactive 3D Models

by   Samuel Reinders, et al.

While developments in 3D printing have opened up opportunities for improved access to graphical information for people who are blind or have low vision (BLV), they can provide only limited detailed and contextual information. Interactive 3D printed models (I3Ms) that provide audio labels and/or a conversational agent interface potentially overcome this limitation. We conducted a Wizard-of-Oz exploratory study to uncover the multi-modal interaction techniques that BLV people would like to use when exploring I3Ms, and investigated their attitudes towards different levels of model agency. These findings informed the creation of an I3M prototype of the solar system. A second user study with this model revealed a hierarchy of interaction, with BLV users preferring tactile exploration, followed by touch gestures to trigger audio labels, and then natural language to fill in knowledge gaps and confirm understanding.



There are no comments yet.


page 4

page 7


StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible

Blind people frequently encounter inaccessible dynamic touchscreens in t...

A Multimodal Assistive System for Helping Visually Impaired in Social Interactions

Access to non-verbal cues in social interactions is vital for people wit...

Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning

Interactive reinforcement learning (IRL) extends traditional reinforceme...

VibEmoji: Exploring User-authoring Multi-modal Emoticons in Social Communication

Emoticons are indispensable in online communications. With users' growin...

A Multifunction Printer CUI for the Blind

Advances in interface design using touch surfaces creates greater obstac...

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Visualization requirements in Forensic Lucid have to do with different l...

Bringing Cognitive Augmentation to Web Browsing Accessibility

In this paper we explore the opportunities brought by cognitive augmenta...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.