Lexicon-Free Fingerspelling Recognition from Video: Data, Models, and Signer Adaptation

09/26/2016
by   Taehwan Kim, et al.
0

We study the problem of recognizing video sequences of fingerspelled letters in American Sign Language (ASL). Fingerspelling comprises a significant but relatively understudied part of ASL. Recognizing fingerspelling is challenging for a number of reasons: It involves quick, small motions that are often highly coarticulated; it exhibits significant variation between signers; and there has been a dearth of continuous fingerspelling data collected. In this work we collect and annotate a new data set of continuous fingerspelling videos, compare several types of recognizers, and explore the problem of signer variation. Our best-performing models are segmental (semi-Markov) conditional random fields using deep neural network-based features. In the signer-dependent setting, our recognizers achieve up to about 92 multi-signer setting is much more challenging, but with neural network adaptation we achieve up to 83

READ FULL TEXT

page 3

page 7

page 11

page 17

page 22

page 23

research
08/30/2016

American Sign Language fingerspelling recognition from video: Methods for unrestricted recognition and signer-independence

In this thesis, we study the problem of recognizing video sequences of f...
research
02/13/2016

Signer-independent Fingerspelling Recognition with Deep Neural Network Adaptation

We study the problem of recognition of fingerspelled letter sequences in...
research
10/26/2018

American Sign Language fingerspelling recognition in the wild

We address the problem of American Sign Language fingerspelling recognit...
research
06/07/2019

Recognizing American Sign Language Manual Signs from RGB-D Videos

In this paper, we propose a 3D Convolutional Neural Network (3DCNN) base...
research
12/20/2014

Incremental Adaptation Strategies for Neural Network Language Models

It is today acknowledged that neural network language models outperform ...
research
06/06/2013

Recognition of Indian Sign Language in Live Video

Sign Language Recognition has emerged as one of the important area of re...
research
07/03/2021

A Data-Driven Method for Recognizing Automated Negotiation Strategies

Understanding an opponent agent helps in negotiating with it. Existing w...

Please sign up or login with your details

Forgot password? Click here to reset