Scene-Aware Audio for 360 Videos

05/12/2018
by   Dingzeyu Li, et al.
0

Although 360 cameras ease the capture of panoramic footage, it remains challenging to add realistic 360 audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360 videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.

READ FULL TEXT

page 1

page 5

page 10

page 11

research
11/14/2019

Scene-Aware Audio Rendering via Deep Acoustic Analysis

We present a new method to capture the acoustic characteristics of real-...
research
11/21/2021

Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video

Binaural audio provides human listeners with an immersive spatial sound ...
research
12/02/2022

Relative Acoustic Features for Distance Estimation in Smart-Homes

Any audio recording encapsulates the unique fingerprint of the associate...
research
06/29/2023

Evaluation of Virtual Acoustic Environments with Different Acoustic Level of Detail

Virtual acoustic environments enable the creation and simulation of real...
research
09/06/2023

Leveraging Geometrical Acoustic Simulations of Spatial Room Impulse Responses for Improved Sound Event Detection and Localization

As deeper and more complex models are developed for the task of sound ev...
research
10/24/2017

Inferring Room Semantics Using Acoustic Monitoring

Having knowledge of the environmental context of the user i.e. the knowl...
research
09/16/2018

Cocktails, but no party: multipath-enabled private audio

We describe a private audio messaging system that uses echoes to unscram...

Please sign up or login with your details

Forgot password? Click here to reset