Cordelia Schmid

research

∙ 08/28/2023

CoVR: Learning Composed Video Retrieval from Web Video Captions

Composed Image Retrieval (CoIR) has recently gained popularity as a task...

0 Lucas Ventura, et al. ∙

research

∙ 08/24/2023

POCO: 3D Pose and Shape Estimation with Confidence

The regression of 3D Human Pose and Shape (HPS) from an image is becomin...

0 Sai Kumar Dwivedi, et al. ∙

research

∙ 08/21/2023

UnLoc: A Unified Framework for Video Localization Tasks

While large-scale image-text pretrained models such as CLIP have been us...

0 Shen Yan, et al. ∙

research

∙ 08/10/2023

Object Goal Navigation with Recursive Implicit Maps

Object goal navigation aims to navigate an agent to locations of a given...

0 Shizhe Chen, et al. ∙

research

∙ 07/28/2023

Robust Visual Sim-to-Real Transfer for Robotic Manipulation

Learning visuomotor policies in simulation is much safer and cheaper tha...

0 Ricardo Garcia, et al. ∙

research

∙ 07/17/2023

Does Visual Pretraining Help End-to-End Reasoning?

We aim to investigate whether end-to-end learning of visual reasoning ca...

0 Chen Sun, et al. ∙

research

∙ 06/20/2023

Dense Video Object Captioning from Disjoint Supervision

We propose a new task and model for dense video object captioning – dete...

0 Xingyi Zhou, et al. ∙

research

∙ 06/20/2023

How can objects help action recognition?

Current state-of-the-art video models process a video clip as a long seq...

0 Xingyi Zhou, et al. ∙

research

∙ 06/13/2023

AVIS: Autonomous Visual Information Seeking with Large Language Models

In this paper, we propose an autonomous information seeking visual quest...

0 Ziniu Hu, et al. ∙

research

∙ 06/12/2023

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts

The visual classification performance of vision-language models such as ...

0 Karsten Roth, et al. ∙

research

∙ 06/12/2023

Retrieval-Enhanced Contrastive Vision-Text Models

Contrastive image-text models such as CLIP form the building blocks of m...

0 Ahmet Iscen, et al. ∙

research

∙ 06/08/2023

Modular Visual Question Answering via Code Generation

We present a framework that formulates visual question answering as modu...

6 Sanjay Subramanian, et al. ∙

research

∙ 05/10/2023

Learning Video-Conditioned Policies for Unseen Manipulation Tasks

The ability to specify robot commands by a non-expert user is critical f...

0 Elliot Chane-Sane, et al. ∙

research

∙ 04/24/2023

End-to-End Spatio-Temporal Action Localisation with Video Transformers

The most performant spatio-temporal action localisation models use exter...

0 Alexey Gritsenko, et al. ∙

research

∙ 04/24/2023

gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction

Signed distance functions (SDFs) is an attractive framework that has rec...

0 Zerui Chen, et al. ∙

research

∙ 04/13/2023

Verbs in Action: Improving verb understanding in video-language models

Understanding verbs is crucial to modelling how people and objects inter...

4 Liliane Momeni, et al. ∙

research

∙ 04/13/2023

Contact Models in Robotics: a Comparative Analysis

Physics simulation is ubiquitous in robotics. Whether in model-based app...

0 Quentin Le Lidec, et al. ∙

research

∙ 04/11/2023

Improving Image Recognition by Retrieving from Web-Scale Image-Text Data

Retrieval augmented models are becoming increasingly popular for compute...

0 Ahmet Iscen, et al. ∙

research

∙ 04/06/2023

Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval

Cross-modal retrieval methods are the preferred tool to search databases...

0 Jae Myung Kim, et al. ∙

research

∙ 04/04/2023

Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification

Due to the expensive costs of collecting labels in multi-label classific...

0 Youngwook Kim, et al. ∙

research

∙ 03/29/2023

AVFormer: Injecting Vision into Frozen Speech Models for Zero-Shot AV-ASR

Audiovisual automatic speech recognition (AV-ASR) aims to improve the ro...

5 Paul Hongsuck Seo, et al. ∙

research

∙ 02/27/2023

Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning

In this work, we introduce Vid2Seq, a multi-modal single-stage dense eve...

5 Antoine Yang, et al. ∙

research

∙ 12/20/2022

Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation

One of the major challenges of machine translation (MT) is ambiguity, wh...

0 Matthieu Futeral, et al. ∙

research

∙ 12/10/2022

REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory

In this paper, we propose an end-to-end Retrieval-Augmented Visual Langu...

0 Ziniu Hu, et al. ∙

research

∙ 12/09/2022

Audiovisual Masked Autoencoders

Can we leverage the audiovisual information already present in video to ...

0 Mariana-Iuliana Georgescu, et al. ∙

research

∙ 12/05/2022

Location-Aware Self-Supervised Transformers

Pixel-level labels are particularly expensive to acquire. Hence, pretrai...

0 Mathilde Caron, et al. ∙

research

∙ 11/25/2022

WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Prediction

This paper presents WALDO (WArping Layer-Decomposed Objects), a novel ap...

0 Guillaume Le Moing, et al. ∙

research

∙ 11/18/2022

AVATAR submission to the Ego4D AV Transcription Challenge

In this report, we describe our submission to the Ego4D AudioVisual (AV)...

6 Paul Hongsuck Seo, et al. ∙

research

∙ 11/17/2022

Language Conditioned Spatial Relation Reasoning for 3D Object Grounding

Localizing objects in 3D scenes based on natural language requires under...

0 Shizhe Chen, et al. ∙

research

∙ 11/16/2022

Learning Reward Functions for Robotic Manipulation by Observing Humans

Observing a human demonstrator manipulate objects provides a rich, scala...

1 Minttu Alakuijala, et al. ∙

research

∙ 10/10/2022

A Memory Transformer Network for Incremental Learning

We study class-incremental learning, a training setup in which new class...

0 Ahmet Iscen, et al. ∙

research

∙ 09/19/2022

Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control

Reinforcement learning (RL) and trajectory optimization (TO) present str...

11 Quentin Le Lidec, et al. ∙

research

∙ 09/11/2022

Instruction-driven history-aware policies for robotic manipulations

In human environments, robots are expected to accomplish a variety of ma...

6 Pierre-Louis Guhur, et al. ∙

research

∙ 08/24/2022

Learning from Unlabeled 3D Environments for Vision-and-Language Navigation

In vision-and-language navigation (VLN), an embodied agent is required t...

17 Shizhe Chen, et al. ∙

research

∙ 08/14/2022

TL;DW? Summarizing Instructional Videos with Task Relevance Cross-Modal Saliency

YouTube users looking for instructions for a specific task may spend a l...

2 Medhini Narasimhan, et al. ∙

research

∙ 07/26/2022

AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction

Recent work achieved impressive progress towards joint reconstruction of...

4 Zerui Chen, et al. ∙

research

∙ 07/08/2022

Beyond Transfer Learning: Co-finetuning for Action Localisation

Transfer learning is the predominant paradigm for training deep networks...

9 Anurag Arnab, et al. ∙

research

∙ 06/23/2022

Augmenting differentiable physics with randomized smoothing

In the past few years, following the differentiable programming paradigm...

0 Quentin Le Lidec, et al. ∙

research

∙ 06/20/2022

M M Mix: A Multimodal Multiview Transformer Ensemble

This report describes the approach behind our winning solution to the 20...

2 Xuehan Xiong, et al. ∙

research

∙ 06/16/2022

Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

Video question answering (VideoQA) is a complex task that requires diver...

8 Antoine Yang, et al. ∙

research

∙ 06/15/2022

AVATAR: Unconstrained Audiovisual Speech Recognition

Audio-visual automatic speech recognition (AV-ASR) is an extension of AS...

8 Valentin Gabeur, et al. ∙

research

∙ 05/10/2022

Learning to Answer Visual Questions from Web Videos

Recent methods for visual question answering rely on large-scale annotat...

7 Antoine Yang, et al. ∙

research

∙ 05/10/2022

Weakly-supervised segmentation of referring expressions

Visual grounding localizes regions (boxes or segments) in the image corr...

10 Robin Strudel, et al. ∙

research

∙ 04/20/2022

Assembly Planning from Observations under Physical Constraints

This paper addresses the problem of copying an unknown assembly of primi...

4 Thomas Chabal, et al. ∙

research

∙ 04/01/2022

Learning Audio-Video Modalities from Image Captions

A major challenge in text-video and text-audio retrieval is the lack of ...

3 Arsha Nagrani, et al. ∙

research

∙ 03/30/2022

TubeDETR: Spatio-Temporal Video Grounding with Transformers

We consider the problem of localizing a spatio-temporal tube in a video ...

8 Antoine Yang, et al. ∙

research

∙ 03/08/2022

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Optimal control (OC) algorithms such as Differential Dynamic Programming...

0 Quentin Le Lidec, et al. ∙

research

∙ 02/28/2022

The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields

Both a good understanding of geometrical concepts and a broad familiarit...

7 Pia Bideau, et al. ∙

research

∙ 02/23/2022

Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation

Following language instructions to navigate in unseen environments is a ...

35 Shizhe Chen, et al. ∙

research

∙ 02/04/2022

Learning with Neighbor Consistency for Noisy Labels

Recent advances in deep learning have relied on large, labelled datasets...

4 Ahmet Iscen, et al. ∙

Cordelia Schmid

Featured Co-authors