research
∙
07/27/2023
Self-Supervised Visual Acoustic Matching
Acoustic matching aims to re-synthesize an audio clip to sound as if it ...
share
research
∙
07/17/2023
Video-Mined Task Graphs for Keystep Recognition in Instructional Videos
Procedural activity understanding requires perceiving human actions in t...
research
∙
07/10/2023
Learning Spatial Features from Audio-Visual Correspondence in Egocentric Videos
We propose a self-supervised method for learning representations based o...
research
∙
06/28/2023
SpotEM: Efficient Video Search for Episodic Memory
The goal in episodic memory (EM) is to search a long egocentric video to...
research
∙
06/15/2023
Single-Stage Visual Query Localization in Egocentric Videos
Visual Query Localization on long-form egocentric videos requires spatio...
research
∙
06/08/2023
Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment
The egocentric and exocentric viewpoints of a human activity look dramat...
research
∙
02/03/2023
Egocentric Video Task Translation @ Ego4D Challenge 2022
This technical report describes the EgoTask Translation approach that ex...
research
∙
01/20/2023
Novel-View Acoustic Synthesis
We introduce the novel-view acoustic synthesis (NVAS) task: given the si...
research
∙
01/18/2023
A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
Despite the advancement of machine learning techniques in recent years, ...
research
∙
01/09/2023
EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset
Visual object tracking is a key component to many egocentric vision prob...
research
∙
01/05/2023
HierVL: Learning Hierarchical Video-Language Embeddings
Video-language embeddings are a promising avenue for injecting semantics...
research
∙
01/05/2023
What You Say Is What You Show: Visual Narration Detection in Instructional Videos
Narrated "how-to" videos have emerged as a promising data source for a w...
research
∙
01/05/2023
EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding
Recent advances in egocentric video understanding models are promising, ...
research
∙
01/04/2023
Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations
Can conversational videos captured from multiple egocentric viewpoints r...
research
∙
01/02/2023
NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory
Searching long egocentric videos with natural language queries (NLQ) has...
research
∙
12/13/2022
Egocentric Video Task Translation
Different video understanding tasks are typically treated in isolation, ...
research
∙
07/22/2022
Egocentric scene context for human-centric environment understanding from video
First-person video highlights a camera-wearer's activities in the contex...
research
∙
06/16/2022
SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning
We introduce SoundSpaces 2.0, a platform for on-the-fly geometry-based a...
research
∙
06/08/2022
Few-Shot Audio-Visual Learning of Environment Acoustics
Room impulse response (RIR) functions capture how the surrounding physic...
research
∙
02/14/2022
Visual Acoustic Matching
We introduce the visual acoustic matching task, in which an audio clip i...
research
∙
02/05/2022
Zero Experience Required: Plug Play Modular Transfer Learning for Semantic Visual Navigation
In reinforcement learning for visual navigation, it is common to develop...
research
∙
02/02/2022
Active Audio-Visual Separation of Dynamic Sound Sources
We explore active audio-visual separation for dynamic sound sources, whe...
research
∙
02/01/2022
DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video
Dexterous multi-fingered robotic hands have a formidable action space, y...
research
∙
01/25/2022
PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
State-of-the-art approaches to ObjectGoal navigation rely on reinforceme...
research
∙
11/21/2021
Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video
Binaural audio provides human listeners with an immersive spatial sound ...
research
∙
10/14/2021
Shaping embodied agent behavior with activity-context priors from egocentric video
Complex physical tasks entail a sequence of object interactions, each wi...
research
∙
06/14/2021
Learning Audio-Visual Dereverberation
Reverberation from audio reflecting off surfaces and objects in the envi...
research
∙
06/03/2021
Anticipative Video Transformer
We propose Anticipative Video Transformer (AVT), an end-to-end attention...
research
∙
05/20/2021
Egocentric Activity Recognition and Localization on a 3D Map
Given a video captured from a first person perspective and recorded in a...
research
∙
05/15/2021
Move2Hear: Active Audio-Visual Source Separation
We introduce the active audio-visual source separation problem, where an...
research
∙
04/16/2021
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos
We introduce an approach for pre-training egocentric video models using ...
research
∙
04/01/2021
Multiview Pseudo-Labeling for Semi-supervised Learning from Video
We present a multiview pseudo-labeling approach to video learning, a nov...
research
∙
02/03/2021
Environment Predictive Coding for Embodied Agents
We introduce environment predictive coding, a self-supervised approach t...
research
∙
02/02/2021
From Culture to Clothing: Discovering the World Events Behind A Century of Fashion Images
Fashion is intertwined with external cultural factors, but identifying t...
research
∙
01/08/2021
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
We introduce a new approach for audio-visual speech separation. Given a ...
research
∙
12/21/2020
Semantic Audio-Visual Navigation
Recent work on audio-visual navigation assumes a constantly-sounding tar...
research
∙
12/04/2020
Discovering Underground Maps from Fashion
The fashion sense – meaning the clothing styles people wear – in a geogr...
research
∙
11/17/2020
Modeling Fashion Influence from Photos
The evolution of clothing styles and their migration across the world is...
research
∙
09/03/2020
Dexterous Robotic Grasping with Object-Centric Visual Affordances
Dexterous robotic hands are appealing for their agility and human-like m...
research
∙
08/21/2020
Learning to Set Waypoints for Audio-Visual Navigation
In audio-visual navigation, an agent intelligently travels through a com...
research
∙
08/21/2020
Occupancy Anticipation for Efficient Exploration and Navigation
State-of-the-art navigation methods leverage a spatial memory to general...
research
∙
08/21/2020
Learning Affordance Landscapes forInteraction Exploration in 3D Environments
Embodied agents operating in human spaces must be able to master how the...
research
∙
06/29/2020
Learning Patterns of Tourist Movement and Photography from Geotagged Photos at Archaeological Heritage Sites in Cuzco, Peru
The popularity of media sharing platforms in recent decades has provided...
research
∙
05/04/2020
VisualEchoes: Spatial Image Representation Learning through Echolocation
Several animal species (e.g., bats, dolphins, and whales) and even visua...
research
∙
04/03/2020
From Paris to Berlin: Discovering Fashion Style Influences Around the World
The evolution of clothing styles and their migration across the world is...
research
∙
01/23/2020
Audiovisual SlowFast Networks for Video Recognition
We present Audiovisual SlowFast Networks, an architecture for integrated...
research
∙
01/14/2020
EGO-TOPO: Environment Affordances from Egocentric Video
First-person video naturally brings the use of a physical environment to...
research
∙
01/09/2020
Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias
Existing models often leverage co-occurrences between objects and their ...
research
∙
01/07/2020
An Exploration of Embodied Visual Exploration
Embodied computer vision considers perception for robots in general, uns...
research
∙
12/24/2019
