DeepAI AI Chat
Log In Sign Up

Building Korean Sign Language Augmentation (KoSLA) Corpus with Data Augmentation Technique

07/12/2022
by   Changnam An, et al.
Yonsei University
0

We present an efficient framework of corpus for sign language translation. Aided with a simple but dramatic data augmentation technique, our method converts text into annotated forms with minimum information loss. Sign languages are composed of manual signals, non-manual signals, and iconic features. According to professional sign language interpreters, non-manual signals such as facial expressions and gestures play an important role in conveying exact meaning. By considering the linguistic features of sign language, our proposed framework is a first and unique attempt to build a multimodal sign language augmentation corpus (hereinafter referred to as the KoSLA corpus) containing both manual and non-manual modalities. The corpus we built demonstrates confident results in the hospital context, showing improved performance with augmented datasets. To overcome data scarcity, we resorted to data augmentation techniques such as synonym replacement to boost the efficiency of our translation model and available data, while maintaining grammatical and semantic structures of sign language. For the experimental support, we verify the effectiveness of data augmentation technique and usefulness of our corpus by performing a translation task between normal sentences and sign language annotations on two tokenizers. The result was convincing, proving that the BLEU scores with the KoSLA corpus were significant.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/02/2022

Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22

This paper describes the system developed at the Universitat Politècnica...
04/12/2022

Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation

Sign language recognition and translation first uses a recognition modul...
03/31/2023

Traffic Sign Recognition Dataset and Data Augmentation

Although there are many datasets for traffic sign classification, there ...
08/24/2020

Global-local Enhancement Network for NMFs-aware Sign Language Recognition

Sign language recognition (SLR) is a challenging problem, involving comp...
05/18/2023

Cross-modality Data Augmentation for End-to-End Sign Language Translation

End-to-end sign language translation (SLT) aims to convert sign language...
05/01/2020

Recognizing American Sign Language Nonmanual Signal Grammar Errors in Continuous Videos

As part of the development of an educational tool that can help students...
09/29/2022

PerSign: Personalized Bangladeshi Sign Letters Synthesis

Bangladeshi Sign Language (BdSL) - like other sign languages - is tough ...