Phoneme-to-viseme mappings: the good, the bad, and the ugly

05/08/2018
by   Helen L Bear, et al.
0

Visemes are the visual equivalent of phonemes. Although not precisely defined, a working definition of a viseme is "a set of phonemes which have identical appearance on the lips". Therefore a phoneme falls into one viseme class but a viseme may represent many phonemes: a many to one mapping. This mapping introduces ambiguity between phonemes when using viseme classifiers. Not only is this ambiguity damaging to the performance of audio-visual classifiers operating on real expressive speech, there is also considerable choice between possible mappings. In this paper we explore the issue of this choice of viseme-to-phoneme map. We show that there is definite difference in performance between viseme-to-phoneme mappings and explore why some maps appear to work better than others. We also devise a new algorithm for constructing phoneme-to-viseme mappings from labeled speech data. These new visemes, `Bear' visemes, are shown to perform better than previously known units.

READ FULL TEXT

page 11

page 12

page 17

research
10/03/2017

Which phoneme-to-viseme maps best improve visual-only computer lip-reading?

A critical assumption of all current visual speech recognition systems i...
research
06/16/2021

Latent Mappings: Generating Open-Ended Expressive Mappings Using Variational Autoencoders

In many contexts, creating mappings for gestural interactions can form p...
research
01/25/2022

Characterizations and constructions of n-to-1 mappings over finite fields

n-to-1 mappings have wide applications in many areas, especially in cryp...
research
07/12/2018

Novel Method for Multi-Dimensional Mapping of Higher Order Modulations for BICM-ID Over Rayleigh Fading Channels

Multi-dimensional (MD) mapping offers more flexibility in mapping design...
research
02/27/2018

Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data

Learning inter-domain mappings from unpaired data can improve performanc...
research
07/12/2019

Equiprobable mappings in weighted constraint grammars

We show that MaxEnt is so rich that it can distinguish between any two d...
research
09/14/2023

Proximal Bellman mappings for reinforcement learning and their application to robust adaptive filtering

This paper aims at the algorithmic/theoretical core of reinforcement lea...

Please sign up or login with your details

Forgot password? Click here to reset