Sign and Search: Sign Search Functionality for Sign Language Lexica

07/28/2021
by   Manolis Fragkiadakis, et al.
0

Sign language lexica are a useful resource for researchers and people learning sign languages. Current implementations allow a user to search a sign either by its gloss or by selecting its primary features such as handshape and location. This study focuses on exploring a reverse search functionality where a user can sign a query sign in front of a webcam and retrieve a set of matching signs. By extracting different body joints combinations (upper body, dominant hand's arm and wrist) using the pose estimation framework OpenPose, we compare four techniques (PCA, UMAP, DTW and Euclidean distance) as distance metrics between 20 query signs, each performed by eight participants on a 1200 sign lexicon. The results show that UMAP and DTW can predict a matching sign with an 80% and 71% accuracy respectively at the top-20 retrieved signs using the movement of the dominant hand arm. Using DTW and adding more sign instances from other participants in the lexicon, the accuracy can be raised to 90% at the top-10 ranking. Our results suggest that our methodology can be used with no training in any sign language lexicon regardless of its size.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2020

Score-level Multi Cue Fusion for Sign Language Recognition

Sign Languages are expressed through hand and upper body gestures as wel...
research
02/11/2023

Improving Sign Recognition with Phonology

We use insights from research on American Sign Language (ASL) phonology ...
research
12/22/2020

Speak with signs: Active learning platform for Greek Sign Language, English Sign Language, and their translation

Sign Language is used to facilitate the communication between Deaf and n...
research
07/12/2023

Enhancing Portuguese Sign Language Animation with Dynamic Timing and Mouthing

Current signing avatars are often described as unnatural as they cannot ...
research
03/19/2023

On the Importance of Signer Overlap for Sign Language Detection

Sign language detection, identifying if someone is signing or not, is be...
research
05/24/2022

Classification of Phonological Parameters in Sign Languages

Signers compose sign language phonemes that enable communication by comb...
research
02/26/2016

Victory Sign Biometric for Terrorists Identification

Covering the face and all body parts, sometimes the only evidence to ide...

Please sign up or login with your details

Forgot password? Click here to reset