The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and a Tongue-Drive

02/01/2018

∙

by   Fu-Jen Chu, et al.

∙

0

∙

share

A human-in-the-loop system is proposed to enable collaborative manipulation tasks for person with physical disabilities. Studies have shown the cognitive burden of subject is reduced with increased autonomy of assistive system. Our framework is aim to communicate high-level intent from subject and generate desired manipulations. We elaborate a framework by incorporating a tongue-drive system and a 7 DoF robotic arm through an virtual interface. The assistive system processes sensor input for interpreting users environment, and the subject provides an ego-centric visual feedback via interface to guide action loop for achieving the tasks. Extensive experiments are performed on our framework and we show our coupled feedback loops is able to effectively simplify complex manipulation tasks.