Log In Sign Up

Lip reading using external viseme decoding

by   Javad Peymanfard, et al.

Lip-reading is the operation of recognizing speech from lip movements. This is a difficult task because the movements of the lips when pronouncing the words are similar for some of them. Viseme is used to describe lip movements during a conversation. This paper aims to show how to use external text data (for viseme-to-character mapping) by dividing video-to-character into two stages, namely converting video to viseme, and then converting viseme to character by using separate models. Our proposed method improves word error rate by 4% compared to the normal sequence to sequence lip-reading model on the BBC-Oxford Lip Reading Sentences 2 (LRS2) dataset.


DualLip: A System for Joint Lip Reading and Generation

Lip reading aims to recognize text from talking lip, while lip generatio...

Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading

Lip-reading aims to infer the speech content from the lip movement seque...

Character-based Surprisal as a Model of Human Reading in the Presence of Errors

Intuitively, human readers cope easily with errors in text; typos, missp...

Deep Lip Reading: a comparison of models and an online application

The goal of this paper is to develop state-of-the-art models for lip rea...

A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading

Lip reading aims at decoding texts from the movement of a speaker's mout...

Visual Passwords Using Automatic Lip Reading

This paper presents a visual passwords system to increase security. The ...

Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers

Lip reading has witnessed unparalleled development in recent years thank...