DeepAI AI Chat
Log In Sign Up

Global-local Enhancement Network for NMFs-aware Sign Language Recognition

by   Hezhen Hu, et al.

Sign language recognition (SLR) is a challenging problem, involving complex manual features, i.e., hand gestures, and fine-grained non-manual features (NMFs), i.e., facial expression, mouth shapes, etc. Although manual features are dominant, non-manual features also play an important role in the expression of a sign word. Specifically, many sign words convey different meanings due to non-manual features, even though they share the same hand gestures. This ambiguity introduces great challenges in the recognition of sign words. To tackle the above issue, we propose a simple yet effective architecture called Global-local Enhancement Network (GLE-Net), including two mutually promoted streams towards different crucial aspects of SLR. Of the two streams, one captures the global contextual relationship, while the other models the discriminative fine-grained cues. Moreover, due to the lack of datasets explicitly focusing on this kind of features, we introduce the first non-manual features-aware isolated Chinese sign language dataset (NMFs-CSL) with a total vocabulary size of 1,067 sign words in daily life. Extensive experiments on NMFs-CSL and SLR500 datasets demonstrate the effectiveness of our method.


page 2

page 5

page 12


Temporal Accumulative Features for Sign Language Recognition

In this paper, we propose a set of features called temporal accumulative...

Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions

In recent years, Word-level Sign Language Recognition (WSLR) research ha...

Quantitative Survey of the State of the Art in Sign Language Recognition

This work presents a meta study covering around 300 published sign langu...

Building Korean Sign Language Augmentation (KoSLA) Corpus with Data Augmentation Technique

We present an efficient framework of corpus for sign language translatio...

Transferring Cross-domain Knowledge for Video Sign Language Recognition

Word-level sign language recognition (WSLR) is a fundamental task in sig...

A Fine-Grained Visual Attention Approach for Fingerspelling Recognition in the Wild

Fingerspelling in sign language has been the means of communicating tech...

Evaluation Of Hidden Markov Models Using Deep CNN Features In Isolated Sign Recognition

Isolated sign recognition from video streams is a challenging problem du...