Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition

11/01/2022
by   Youngjoon Jang, et al.
0

The goal of this work is background-robust continuous sign language recognition. Most existing Continuous Sign Language Recognition (CSLR) benchmarks have fixed backgrounds and are filmed in studios with a static monochromatic background. However, signing is not limited only to studios in the real world. In order to analyze the robustness of CSLR models under background shifts, we first evaluate existing state-of-the-art CSLR models on diverse backgrounds. To synthesize the sign videos with a variety of backgrounds, we propose a pipeline to automatically generate a benchmark dataset utilizing existing CSLR benchmarks. Our newly constructed benchmark dataset consists of diverse scenes to simulate a real-world environment. We observe even the most recent CSLR method cannot recognize glosses well on our new dataset with changed backgrounds. In this regard, we also propose a simple yet effective training scheme including (1) background randomization and (2) feature disentanglement for CSLR models. The experimental results on our dataset demonstrate that our method generalizes well to other unseen background data with minimal additional training images.

READ FULL TEXT

page 2

page 4

page 5

page 10

page 18

page 19

page 22

page 23

research
03/30/2021

Read and Attend: Temporal Localisation in Sign Language Videos

The objective of this work is to annotate sign instances across a broad ...
research
05/29/2020

Learning From Context-Agnostic Synthetic Data

We present a new approach for synthesizing training data given only a si...
research
12/07/2017

CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition

In this paper, we investigate the robustness of traffic sign recognition...
research
11/16/2019

Revisiting Shadow Detection: A New Benchmark Dataset for Complex World

Shadow detection in general photos is a nontrivial problem, due to the c...
research
07/24/2019

Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?

We introduce the problem of zero-shot sign language recognition (ZSSLR),...
research
08/22/2020

Quantitative Survey of the State of the Art in Sign Language Recognition

This work presents a meta study covering around 300 published sign langu...
research
10/08/2022

ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition

Sign language recognition has attracted the interest of researchers in r...

Please sign up or login with your details

Forgot password? Click here to reset