Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions

06/30/2021
by   Mizuki Maruyama, et al.
0

In recent years, Word-level Sign Language Recognition (WSLR) research has gained popularity in the computer vision community, and thus various approaches have been proposed. Among these approaches, the method using I3D network achieves the highest recognition accuracy on large public datasets for WSLR. However, the method with I3D only utilizes appearance information of the upper body of the signers to recognize sign language words. On the other hand, in WSLR, the information of local regions, such as the hand shape and facial expression, and the positional relationship among the body and both hands are important. Thus in this work, we utilized local region images of both hands and face, along with skeletal information to capture local information and the positions of both hands relative to the body, respectively. In other words, we propose a novel multi-stream WSLR framework, in which a stream with local region images and a stream with skeletal information are introduced by extending I3D network to improve the recognition accuracy of WSLR. From the experimental results on WLASL dataset, it is evident that the proposed method has achieved about 15 conventional methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

page 5

page 7

page 9

page 10

09/29/2020

Score-level Multi Cue Fusion for Sign Language Recognition

Sign Languages are expressed through hand and upper body gestures as wel...
08/24/2020

Global-local Enhancement Network for NMFs-aware Sign Language Recognition

Sign language recognition (SLR) is a challenging problem, involving comp...
10/24/2019

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Vision-based sign language recognition aims at helping the hearing-impai...
08/30/2012

Benchmarking recognition results on word image datasets

We have benchmarked the maximum obtainable recognition accuracy on vario...
11/10/2017

Egocentric Hand Detection Via Dynamic Region Growing

Egocentric videos, which mainly record the activities carried out by the...
10/09/2016

Spatial Relationship Based Features for Indian Sign Language Recognition

In this paper, the task of recognizing signs made by hearing impaired pe...
11/16/2017

Grammatical facial expression recognition using customized deep neural network architecture

This paper proposes to expand the visual understanding capacity of compu...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.