American Sign Language Recognition Using RF Sensing

09/02/2020
by   Sevgi Z. Gurbuz, et al.
0

Many technologies for human-computer interaction have been designed for hearing individuals and depend upon vocalized speech, precluding users of American Sign Language (ASL) in the Deaf community from benefiting from these advancements. While great strides have been made in ASL recognition with video or wearable gloves, the use of video in homes has raised privacy concerns, while wearable gloves severely restrict movement and infringe on daily life. Methods: This paper proposes the use of RF sensors for HCI applications serving the Deaf community. A multi-frequency RF sensor network is used to acquire non-invasive, non-contact measurements of ASL signing irrespective of lighting conditions. The unique patterns of motion present in the RF data due to the micro-Doppler effect are revealed using time-frequency analysis with the Short-Time Fourier Transform. Linguistic properties of RF ASL data are investigated using machine learning (ML). Results: The information content, measured by fractal complexity, of ASL signing is shown to be greater than that of other upper body activities encountered in daily living. This can be used to differentiate daily activities from signing, while features from RF data show that imitation signing by non-signers is 99% differentiable from native ASL signing. Feature-level fusion of RF sensor network data is used to achieve 72.5% accuracy in classification of 20 native ASL signs. Implications: RF sensing can be used to study dynamic linguistic properties of ASL and design Deaf-centric smart environments for non-invasive, remote recognition of ASL. ML algorithms should be benchmarked on native, not imitation, ASL data.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

research
10/03/2022

Smart-Badge: A wearable badge with multi-modal sensors for kitchen activity recognition

Human health is closely associated with their daily behavior and environ...
research
01/09/2023

EMAHA-DB1: A New Upper Limb sEMG Dataset for Classification of Activities of Daily Living

In this paper, we present electromyography analysis of human activity - ...
research
10/01/2018

Using Machine Learning to Discern Eruption in Noisy Environments: A Case Study using CO2-driven Cold-Water Geyser in Chimayo, New Mexico

We present an approach based on machine learning (ML) to distinguish eru...
research
10/27/2021

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network

Human Activity Recognition (HAR) plays a critical role in a wide range o...
research
07/29/2020

A Flexible and Modular Body-Machine Interface for Individuals Living with Severe Disabilities

This paper presents a control interface to translate the residual body m...
research
08/25/2020

In-Home Daily-Life Captioning Using Radio Signals

This paper aims to caption daily life –i.e., to create a textual descrip...
research
03/21/2020

RF Sensing for Continuous Monitoring of Human Activities for Home Consumer Applications

Radar for indoor monitoring is an emerging area of research and developm...

Please sign up or login with your details

Forgot password? Click here to reset