DeepAI
Log In Sign Up

Lip reading using external viseme decoding

04/10/2021
by   Javad Peymanfard, et al.
0

Lip-reading is the operation of recognizing speech from lip movements. This is a difficult task because the movements of the lips when pronouncing the words are similar for some of them. Viseme is used to describe lip movements during a conversation. This paper aims to show how to use external text data (for viseme-to-character mapping) by dividing video-to-character into two stages, namely converting video to viseme, and then converting viseme to character by using separate models. Our proposed method improves word error rate by 4% compared to the normal sequence to sequence lip-reading model on the BBC-Oxford Lip Reading Sentences 2 (LRS2) dataset.

READ FULL TEXT
09/12/2020

DualLip: A System for Joint Lip Reading and Generation

Lip reading aims to recognize text from talking lip, while lip generatio...
03/09/2020

Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading

Lip-reading aims to infer the speech content from the lip movement seque...
02/02/2019

Character-based Surprisal as a Model of Human Reading in the Presence of Errors

Intuitively, human readers cope easily with errors in text; typos, missp...
06/15/2018

Deep Lip Reading: a comparison of models and an online application

The goal of this paper is to develop state-of-the-art models for lip rea...
08/14/2019

A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading

Lip reading aims at decoding texts from the movement of a speaker's mout...
09/02/2014

Visual Passwords Using Automatic Lip Reading

This paper presents a visual passwords system to increase security. The ...
11/26/2019

Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers

Lip reading has witnessed unparalleled development in recent years thank...