ClearBuds: Wireless Binaural Earbuds for Learning-Based Speech Enhancement

06/27/2022
by   Ishan Chatterjee, et al.
0

We present ClearBuds, the first hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and background cancellation, operating in real-time and on a mobile phone. Clear-Buds bridges state-of-the-art deep learning for blind audio source separation and in-ear mobile systems by making two key technical contributions: 1) a new wireless earbud design capable of operating as a synchronized, binaural microphone array, and 2) a lightweight dual-channel speech enhancement neural network that runs on a mobile device. Our neural network has a novel cascaded architecture that combines a time-domain conventional neural network with a spectrogram-based frequency masking neural network to reduce the artifacts in the audio output. Results show that our wireless earbuds achieve a synchronization error less than 64 microseconds and our network has a runtime of 21.4 milliseconds on an accompanying mobile phone. In-the-wild evaluation with eight users in previously unseen indoor and outdoor multipath scenarios demonstrates that our neural network generalizes to learn both spatial and acoustic cues to perform noise suppression and background speech removal. In a user-study with 37 participants who spent over 15.4 hours rating 1041 audio samples collected in-the-wild, our system achieves improved mean opinion score and background noise suppression. Project page with demos: https://clearbuds.cs.washington.edu

READ FULL TEXT

page 5

page 9

research
12/21/2022

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement

Prior works on improving speech quality with visual input typically stud...
research
05/06/2021

DBNet: A Dual-branch Network Architecture Processing on Spectrum and Waveform for Single-channel Speech Enhancement

In real acoustic environment, speech enhancement is an arduous task to i...
research
11/02/2021

Reduction of Subjective Listening Effort for TV Broadcast Signals with Recurrent Neural Networks

Listening to the audio of TV broadcast signals can be challenging for he...
research
08/30/2020

Improved Lite Audio-Visual Speech Enhancement

Numerous studies have investigated the effectiveness of audio-visual mul...
research
01/22/2023

Cellular Network Speech Enhancement: Removing Background and Transmission Noise

The primary objective of speech enhancement is to reduce background nois...
research
08/18/2022

Deploying Enhanced Speech Feature Decreased Audio Complaints at SVT Play VOD Service

At Public Service Broadcaster SVT in Sweden, background music and sounds...
research
09/13/2018

Real-Time Lightweight Chaotic Encryption for 5G IoT Enabled Lip-Reading Driven Secure Hearing-Aid

Existing audio-only hearing-aids are known to perform poorly in noisy si...

Please sign up or login with your details

Forgot password? Click here to reset