DeepAI AI Chat
Log In Sign Up

Global-local Enhancement Network for NMFs-aware Sign Language Recognition

08/24/2020
by   Hezhen Hu, et al.
0

Sign language recognition (SLR) is a challenging problem, involving complex manual features, i.e., hand gestures, and fine-grained non-manual features (NMFs), i.e., facial expression, mouth shapes, etc. Although manual features are dominant, non-manual features also play an important role in the expression of a sign word. Specifically, many sign words convey different meanings due to non-manual features, even though they share the same hand gestures. This ambiguity introduces great challenges in the recognition of sign words. To tackle the above issue, we propose a simple yet effective architecture called Global-local Enhancement Network (GLE-Net), including two mutually promoted streams towards different crucial aspects of SLR. Of the two streams, one captures the global contextual relationship, while the other models the discriminative fine-grained cues. Moreover, due to the lack of datasets explicitly focusing on this kind of features, we introduce the first non-manual features-aware isolated Chinese sign language dataset (NMFs-CSL) with a total vocabulary size of 1,067 sign words in daily life. Extensive experiments on NMFs-CSL and SLR500 datasets demonstrate the effectiveness of our method.

READ FULL TEXT

page 2

page 5

page 12

04/02/2020

Temporal Accumulative Features for Sign Language Recognition

In this paper, we propose a set of features called temporal accumulative...
06/30/2021

Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions

In recent years, Word-level Sign Language Recognition (WSLR) research ha...
08/22/2020

Quantitative Survey of the State of the Art in Sign Language Recognition

This work presents a meta study covering around 300 published sign langu...
07/12/2022

Building Korean Sign Language Augmentation (KoSLA) Corpus with Data Augmentation Technique

We present an efficient framework of corpus for sign language translatio...
03/08/2020

Transferring Cross-domain Knowledge for Video Sign Language Recognition

Word-level sign language recognition (WSLR) is a fundamental task in sig...
05/17/2021

A Fine-Grained Visual Attention Approach for Fingerspelling Recognition in the Wild

Fingerspelling in sign language has been the means of communicating tech...
06/19/2020

Evaluation Of Hidden Markov Models Using Deep CNN Features In Isolated Sign Recognition

Isolated sign recognition from video streams is a challenging problem du...