See-Through Captions: Real-Time Captioning on Transparent Display for Deaf and Hard-of-Hearing People

01/27/2021
by   Kenta Yamamoto, et al.
0

Real-time captioning is a useful technique for deaf and hard-of-hearing (DHH) people to talk to hearing people. With the improvement in device performance and the accuracy of automatic speech recognition (ASR), real-time captioning is becoming an important tool for helping DHH people in their daily lives. To realize higher-quality communication and overcome the limitations of mobile and augmented-reality devices, real-time captioning that can be used comfortably while maintaining nonverbal communication and preventing incorrect recognition is required. Therefore, we propose a real-time captioning system that uses a transparent display. In this system, the captions are presented on both sides of the display to address the problem of incorrect ASR, and the highly transparent display makes it possible to see both the body language and the captions.

READ FULL TEXT

page 1

page 3

page 4

research
12/06/2017

Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

The accuracy of Automated Speech Recognition (ASR) technology has improv...
research
09/21/2021

Social, Environmental, and Technical: Factors at Play in the Current Use and Future Design of Small-Group Captioning

Real-time captioning is a critical accessibility tool for many d/Deaf an...
research
02/20/2020

Captioning Images Taken by People Who Are Blind

While an important problem in the vision community is to design algorith...
research
10/20/2022

Communication breakdown: On the low mutual intelligibility between human and neural captioning

We compare the 0-shot performance of a neural caption-based image retrie...
research
02/03/2022

Joint Speech Recognition and Audio Captioning

Speech samples recorded in both indoor and outdoor environments are ofte...
research
08/25/2020

In-Home Daily-Life Captioning Using Radio Signals

This paper aims to caption daily life –i.e., to create a textual descrip...
research
09/18/2019

RTTD-ID: Tracked Captions with Multiple Speakers for Deaf Students

Students who are deaf and hard of hearing cannot hear in class and do no...

Please sign up or login with your details

Forgot password? Click here to reset