FUN! Fast, Universal, Non-Semantic Speech Embeddings

11/09/2020
by   Jacob Peplinski, et al.
0

Learned speech representations can drastically improve performance on tasks with limited labeled data. However, due to their size and complexity, learned representations have limited utility in mobile settings where run-time performance is a significant bottleneck. We propose a class of lightweight universal speech embedding models based on MobileNet that are designed to run efficiently on mobile devices. These embeddings, which encapsulate speech non-semantics and thus can be re-used for several tasks, are trained via knowledge distillation. We show that these embedding models are fast enough to run in real-time on a variety of mobile devices and exhibit negligible performance degradation on most tasks in a recently published benchmark of non-semantic speech tasks. Furthermore, we demonstrate that these representations are useful for mobile health tasks such as mask detection during speech and non-speech human sounds detection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/01/2022

TRILLsson: Distilled Universal Paralinguistic Speech Representations

Recent advances in self-supervision have dramatically improved the quali...
research
09/17/2021

On-device neural speech synthesis

Recent advances in text-to-speech (TTS) synthesis, such as Tacotron and ...
research
02/25/2020

Towards Learning a Universal Non-Semantic Representation of Speech

The ultimate goal of transfer learning is to reduce labeled data require...
research
07/12/2022

Distilled Non-Semantic Speech Embeddings with Binary Neural Networks for Low-Resource Devices

This work introduces BRILLsson, a novel binary neural network-based repr...
research
02/12/2021

Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices

A pruning-based AutoML framework for run-time reconfigurability, namely ...
research
04/27/2020

Compact retail shelf segmentation for mobile deployment

The recent surge of automation in the retail industries has rapidly incr...

Please sign up or login with your details

Forgot password? Click here to reset