Log In Sign Up

OrbTouch: Recognizing Human Touch in Deformable Interfaces with Deep Neural Networks

by   Chris Larson, et al.

User interfaces provide an interactive window between physical and virtual environments. A new concept in the field of human-computer interaction is a soft user interface; a compliant surface that facilitates touch interaction through deformation. Despite the potential of these interfaces, they currently lack a signal processing framework that can efficiently extract information from their deformation. Here we present OrbTouch, a device that uses statistical learning algorithms, based on convolutional neural networks, to map deformations from human touch to categorical labels (i.e., gestures) and touch location using stretchable capacitor signals as inputs. We demonstrate this approach by using the device to control the popular game Tetris. OrbTouch provides a modular, robust framework to interpret deformation in soft media, laying a foundation for new modes of human computer interaction through shape changing solids.


page 2

page 5


A Novel Contactless Human Machine Interface based on Machine Learning

This paper describes a global framework that enables contactless human m...

Sensing and Reconstruction of 3D Deformation on Pneumatic Soft Robots

Real-time proprioception is a challenging problem for soft robots, which...

Soft Robotic Mannequin: Design and Algorithm for Deformation Control

This paper presents a novel soft robotic system for a deformable mannequ...

Natural interaction with traffic control cameras through multimodal interfaces

Human-Computer Interfaces have always played a fundamental role in usabi...

Control Interface Remapping for Bias-Aware Assistive Teleoperation

Users of assistive devices vary in their extent of motor impairment, and...

Psychoacoustic Sonification as User Interface for Human-Machine Interaction

When operating a machine, the operator needs to know some spatial relati...

MuA-Ori: Multimodal Actuated Origami

Recently, inflatable elements integrated in robotics systems have enable...