Enabling Real-time On-chip Audio Super Resolution for Bone Conduction Microphones

12/24/2021
by   Yuang Li, et al.
0

Voice communication using the air conduction microphone in noisy environments suffers from the degradation of speech audibility. Bone conduction microphones (BCM) are robust against ambient noises but suffer from limited effective bandwidth due to their sensing mechanism. Although existing audio super resolution algorithms can recover the high frequency loss to achieve high-fidelity audio, they require considerably more computational resources than available in low-power hearable devices. This paper proposes the first-ever real-time on-chip speech audio super resolution system for BCM. To accomplish this, we built and compared a series of lightweight audio super resolution deep learning models. Among all these models, ATS-UNet is the most cost-efficient because the proposed novel Audio Temporal Shift Module (ATSM) reduces the network's dimensionality while maintaining sufficient temporal features from speech audios. Then we quantized and deployed the ATS-UNet to low-end ARM micro-controller units for real-time embedded prototypes. Evaluation results show that our system achieved real-time inference speed on Cortex-M7 and higher quality than the baseline audio super resolution method. Finally, we conducted a user study with ten experts and ten amateur listeners to evaluate our method's effectiveness to human ears. Both groups perceived a significantly higher speech quality with our method when compared to the solutions with the original BCM or air conduction microphone with cutting-edge noise reduction algorithms.

READ FULL TEXT

page 5

page 9

page 14

page 15

research
09/13/2023

AudioSR: Versatile Audio Super-resolution at Scale

Audio super-resolution is a fundamental task that predicts high-frequenc...
research
10/28/2022

Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs

Neural audio super-resolution models are typically trained on low- and h...
research
05/05/2023

AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions

In recent years, videos and images in 720p (HD), 1080p (FHD) and 4K (UHD...
research
11/29/2021

SwiftSRGAN – Rethinking Super-Resolution for Efficient and Real-time Inference

In recent years, there have been several advancements in the task of ima...
research
10/27/2022

Conditioning and Sampling in Variational Diffusion Models for Speech Super-resolution

Recently, diffusion models (DMs) have been increasingly used in audio pr...
research
02/22/2022

FlowSense: Monitoring Airflow in Building Ventilation Systems Using Audio Sensing

Proper indoor ventilation through buildings' heating, ventilation, and a...
research
05/17/2021

Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report

Video super-resolution has recently become one of the most important mob...

Please sign up or login with your details

Forgot password? Click here to reset