Real-time 3-D Mapping with Estimating Acoustic Materials

09/16/2019
by   Taeyoung Kim, et al.
7

This paper proposes a real-time system integrating an acoustic material estimation from visual appearance and an on-the-fly mapping in the 3-dimension. The proposed method estimates the acoustic materials of surroundings in indoor scenes and incorporates them to a 3-D occupancy map, as a robot moves around the environment. To estimate the acoustic material from the visual cue, we apply the state-of-the-art semantic segmentation CNN network based on the assumption that the visual appearance and the acoustic materials have a strong association. Furthermore, we introduce an update policy to handle the material estimations during the online mapping process. As a result, our environment map with acoustic material can be used for sound-related robotics applications, such as sound source localization taking into account various acoustic propagation (e.g., reflection).

READ FULL TEXT

page 1

page 2

page 4

page 5

research
02/02/2023

Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenes

We present an end-to-end binaural audio rendering approach (Listen2Scene...
research
06/24/2022

How to hide your voice: Noise-cancelling bird photography blind

Getting close to birds is a great challenge in wildlife photography. Bir...
research
04/10/2023

In-situ crack and keyhole pore detection in laser directed energy deposition through acoustic signal and deep learning

Cracks and keyhole pores are detrimental defects in alloys produced by l...
research
04/04/2022

Learning Neural Acoustic Fields

Our environment is filled with rich and dynamic acoustic information. Wh...
research
01/17/2019

Using DNNs to Detect Materials in a Room based on Sound Absorption

The materials of surfaces in a room play an important room in shaping th...
research
09/06/2023

Simultaneous Measurement of Multiple Acoustic Attributes Using Structured Periodic Test Signals Including Music and Other Sound Materials

We introduce a general framework for measuring acoustic properties such ...
research
05/18/2022

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

We propose a mesh-based neural network (MESH2IR) to generate acoustic im...

Please sign up or login with your details

Forgot password? Click here to reset