Transformer-based Hand Gesture Recognition via High-Density EMG Signals: From Instantaneous Recognition to Fusion of Motor Unit Spike Trains

11/29/2022
by   Mansooreh Montazerin, et al.
0

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a compact deep learning framework referred to as the CT-HGR, which employs a vision transformer network to conduct hand gesture recognition using highdensity sEMG (HD-sEMG) signals. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. CT-HGR can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the CT-HGR framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the CT-HGR is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed CT-HGR framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23 electrodes and a window size of 250 ms. The CT-HGR achieves accuracy of 89.13 for instantaneous recognition based on a single frame of HD-sEMG image.

READ FULL TEXT

page 1

page 8

page 9

page 11

page 13

page 14

page 21

page 25

research
01/25/2022

ViT-HGR: Vision Transformer-based Hand Gesture Recognition from High Density Surface EMG Signals

Recently, there has been a surge of significant interest on application ...
research
03/28/2022

TraHGR: Few-shot Learning for Hand Gesture Recognition via ElectroMyography

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromy...
research
10/17/2021

Hand Gesture Recognition Using Temporal Convolutions and Attention Mechanism

Advances in biosignal signal processing and machine learning, in particu...
research
10/19/2019

Sensor fusion using EMG and vision for hand gesture classification in mobile applications

The discrimination of human gestures using wearable solutions is extreme...
research
09/25/2021

TEMGNet: Deep Transformer-based Decoding of Upperlimb sEMG for Hand Gestures Recognition

There has been a surge of recent interest in Machine Learning (ML), part...
research
11/07/2022

Novel Muscle Monitoring by Radiomyography(RMG) and Application to Hand Gesture Recognition

Conventional electromyography (EMG) measures the continuous neural activ...
research
11/09/2019

XceptionTime: A Novel Deep Architecture based on Depthwise Separable Convolutions for Hand Gesture Classification

Capitalizing on the need for addressing the existing challenges associat...

Please sign up or login with your details

Forgot password? Click here to reset