research
∙
04/05/2022
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
Objects play a crucial role in our everyday activities. Though multisens...
share
research
∙
02/14/2022
Visual Acoustic Matching
We introduce the visual acoustic matching task, in which an audio clip i...
research
∙
11/21/2021
Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video
Binaural audio provides human listeners with an immersive spatial sound ...
research
∙
09/16/2021
ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations
Multisensory object-centric perception, reasoning, and interaction have ...
research
∙
01/08/2021
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
We introduce a new approach for audio-visual speech separation. Given a ...
research
∙
08/21/2020
Learning to Set Waypoints for Audio-Visual Navigation
In audio-visual navigation, an agent intelligently travels through a com...
research
∙
05/04/2020
VisualEchoes: Spatial Image Representation Learning through Echolocation
Several animal species (e.g., bats, dolphins, and whales) and even visua...
research
∙
12/10/2019
Listen to Look: Action Recognition by Previewing Audio
In the face of the video data deluge, today's expensive clip-level class...
research
∙
04/16/2019
Co-Separating Sounds of Visual Objects
Learning how objects sound from video is challenging, since they often h...
research
∙
12/11/2018
2.5D Visual Sound
Binaural audio provides a listener with 3D sound sensation, allowing a r...
research
∙
04/05/2018
Learning to Separate Object Sounds by Watching Unlabeled Video
Perceiving a scene most fully requires all the senses. Yet modeling how ...
research
∙
12/12/2017
Im2Flow: Motion Hallucination from Static Images for Action Recognition
Existing methods to recognize actions in static images take the images a...
research
∙
12/05/2016
On-Demand Learning for Deep Image Restoration
While machine learning approaches to image restoration offer great promi...
research
∙
12/01/2016