research
∙
08/10/2023
Object Goal Navigation with Recursive Implicit Maps
Object goal navigation aims to navigate an agent to locations of a given...
share
research
∙
07/28/2023
Robust Visual Sim-to-Real Transfer for Robotic Manipulation
Learning visuomotor policies in simulation is much safer and cheaper tha...
research
∙
05/10/2023
Learning Video-Conditioned Policies for Unseen Manipulation Tasks
The ability to specify robot commands by a non-expert user is critical f...
research
∙
04/24/2023
gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction
Signed distance functions (SDFs) is an attractive framework that has rec...
research
∙
04/13/2023
Contact Models in Robotics: a Comparative Analysis
Physics simulation is ubiquitous in robotics. Whether in model-based app...
research
∙
02/27/2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
In this work, we introduce Vid2Seq, a multi-modal single-stage dense eve...
research
∙
12/20/2022
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation
One of the major challenges of machine translation (MT) is ambiguity, wh...
research
∙
12/14/2022
Image Compression with Product Quantized Masked Image Modeling
Recent neural compression methods have been based on the popular hyperpr...
research
∙
11/24/2022
Multi-Task Learning of Object State Changes from Uncurated Videos
We aim to learn to temporally localize object state changes and the corr...
research
∙
11/17/2022
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding
Localizing objects in 3D scenes based on natural language requires under...
research
∙
09/19/2022
Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control
Reinforcement learning (RL) and trajectory optimization (TO) present str...
research
∙
09/11/2022
Instruction-driven history-aware policies for robotic manipulations
In human environments, robots are expected to accomplish a variety of ma...
research
∙
08/24/2022
Learning from Unlabeled 3D Environments for Vision-and-Language Navigation
In vision-and-language navigation (VLN), an embodied agent is required t...
research
∙
07/26/2022
AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction
Recent work achieved impressive progress towards joint reconstruction of...
research
∙
06/23/2022
Augmenting differentiable physics with randomized smoothing
In the past few years, following the differentiable programming paradigm...
research
∙
06/16/2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
Video question answering (VideoQA) is a complex task that requires diver...
research
∙
05/10/2022
Learning to Answer Visual Questions from Web Videos
Recent methods for visual question answering rely on large-scale annotat...
research
∙
05/10/2022
Weakly-supervised segmentation of referring expressions
Visual grounding localizes regions (boxes or segments) in the image corr...
research
∙
03/30/2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers
We consider the problem of localizing a spatio-temporal tube in a video ...
research
∙
03/22/2022
Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos
Human actions often induce changes of object states such as "cutting an ...
research
∙
03/08/2022
Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems
Optimal control (OC) algorithms such as Differential Dynamic Programming...
research
∙
02/23/2022
Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Following language instructions to navigate in unseen environments is a ...
research
∙
12/20/2021
Are Large-scale Datasets Necessary for Self-Supervised Pre-training?
Pre-training models on large scale datasets, like ImageNet, is a standar...
research
∙
11/02/2021
Estimating 3D Motion and Forces of Human-Object Interactions from Internet Videos
In this paper, we introduce a method to automatically reconstruct the 3D...
research
∙
10/25/2021
History Aware Multimodal Transformer for Vision-and-Language Navigation
Vision-and-language navigation (VLN) aims to build autonomous visual age...
research
∙
10/18/2021
Differentiable Rendering with Perturbed Optimizers
Reasoning about 3D scenes from their 2D image projections is one of the ...
research
∙
09/09/2021
Reconstructing and grounding narrated instructional videos in 3D
Narrated instructional videos often show and describe manipulations of s...
research
∙
08/20/2021
Airbert: In-domain Pretraining for Vision-and-Language Navigation
Vision-and-language navigation (VLN) aims to enable embodied agents to n...
research
∙
08/16/2021
Towards unconstrained joint hand-object reconstruction from RGB videos
Our work aims to obtain 3D reconstruction of hands and manipulated objec...
research
∙
07/01/2021
Goal-Conditioned Reinforcement Learning with Imagined Subgoals
Goal-conditioned reinforcement learning endows an agent with a large var...
research
∙
06/17/2021
XCiT: Cross-Covariance Image Transformers
Following their success in natural language processing, transformers hav...
research
∙
05/12/2021
Segmenter: Transformer for Semantic Segmentation
Image segmentation is often ambiguous at the level of individual image p...
research
∙
03/30/2021
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Our objective is language-based search of large-scale image and video da...
research
∙
02/10/2021
Training Vision Transformers for Image Retrieval
Transformers have shown outstanding results for natural language underst...
research
∙
12/01/2020
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Modern approaches to visual question answering require large annotated d...
research
∙
11/13/2020
Learning Object Manipulation Skills via Approximate State Estimation from Real Videos
Humans are adept at learning new tasks by watching a few instructional v...
research
∙
08/25/2020
Learning Obstacle Representations for Neural Motion Planning
Motion planning and obstacle avoidance is a key challenge in robotics ap...
research
∙
08/03/2020
RareAct: A video dataset of unusual interactions
This paper introduces a manually annotated video dataset of unusual acti...
research
∙
04/30/2020
Occlusion resistant learning of intuitive physics from videos
To reach human performance on complex tasks, a key ability for artificia...
research
∙
04/28/2020
Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction
Modeling hand-object manipulations is essential for understanding how hu...
research
∙
04/15/2020
Learning visual policies for building 3D shape categories
Manipulation and assembly tasks require non-trivial planning of actions ...
research
∙
03/29/2020
Learning Interactions and Relationships between Movie Characters
Interactions between people are often governed by their relationships. O...
research
∙
12/13/2019
Action Modifiers: Learning from Adverbs in Instructional Videos
We present a method to learn a representation for adverbs from instructi...
research
∙
12/13/2019
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
Annotating videos is cumbersome, expensive and not scalable. Yet, many s...
research
∙
12/09/2019
Synthetic Humans for Action Recognition from Unseen Viewpoints
Our goal in this work is to improve the performance of human action reco...
research
∙
08/02/2019
Combining learned skills and reinforcement learning for robotic manipulations
Manipulation tasks such as preparing a meal or assembling furniture rema...
research
∙
06/07/2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Learning text-video embeddings usually requires a dataset of video clips...
research
∙
04/23/2019
Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning
In this paper, we address the problem of visually guided rearrangement p...
research
∙
04/21/2019
Deep Metric Learning Beyond Binary Supervision
Metric Learning for visual similarity has mostly adopted binary supervis...
research
∙
04/11/2019