Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis

03/26/2021
by   Nikhil Singh, et al.
0

Measuring the acoustic characteristics of a space is often done by capturing its impulse response (IR), a representation of how a full-range stimulus sound excites it. This is the first work that generates an IR from a single image, which we call Image2Reverb. This IR is then applied to other signals using convolution, simulating the reverberant characteristics of the space shown in the image. Recording these IRs is both time-intensive and expensive, and often infeasible for inaccessible locations. We use an end-to-end neural network architecture to generate plausible audio impulse responses from single images of acoustic environments. We evaluate our method both by comparisons to ground truth data and by human expert evaluation. We demonstrate our approach by generating plausible impulse responses from diverse settings and formats including well known places, musical halls, rooms in paintings, images from animations and computer games, synthetic environments generated from text, panoramic images, and video conference backgrounds.

READ FULL TEXT

page 12

page 13

page 14

page 15

page 17

page 18

page 21

page 23

research
05/18/2022

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

We propose a mesh-based neural network (MESH2IR) to generate acoustic im...
research
10/25/2020

IR-GAN: Room Impulse Response Generator for Speech Augmentation

We present a Generative Adversarial Network (GAN) based room impulse res...
research
06/21/2021

Attention-based cross-modal fusion for audio-visual voice activity detection in musical video streams

Many previous audio-visual voice-related works focus on speech, ignoring...
research
02/14/2022

Visual Acoustic Matching

We introduce the visual acoustic matching task, in which an audio clip i...
research
03/16/2023

DiffIR: Efficient Diffusion Model for Image Restoration

Diffusion model (DM) has achieved SOTA performance by modeling the image...
research
06/21/2021

MeshRIR: A Dataset of Room Impulse Responses on Meshed Grid Points For Evaluating Sound Field Analysis and Synthesis Methods

A new impulse response (IR) dataset called "MeshRIR" is introduced. Curr...
research
01/13/2020

Button Simulation and Design via FDVV Models

Designing a push-button with desired sensation and performance is challe...

Please sign up or login with your details

Forgot password? Click here to reset