Michael S. Ryoo

research

∙ 06/06/2023

Energy-Based Models for Cross-Modal Localization using Convolutional Transformers

We present a novel framework using Energy-Based Models (EBMs) for locali...

0 Alan Wu, et al. ∙

research

∙ 06/01/2023

Active Reinforcement Learning under Limited Visual Observability

In this work, we investigate Active Reinforcement Learning (Active-RL), ...

0 Jinghuan Shang, et al. ∙

research

∙ 04/05/2023

VicTR: Video-conditioned Text Representations for Activity Recognition

Vision-Language models have shown strong performance in the image-domain...

5 Kumara Kahatapitiya, et al. ∙

research

∙ 11/16/2022

Token Turing Machines

We propose Token Turing Machines (TTM), a sequential, autoregressive Tra...

0 Michael S. Ryoo, et al. ∙

research

∙ 10/28/2022

Grafting Vision Transformers

Vision Transformers (ViTs) have recently become the state-of-the-art acr...

0 Jongwoo Park, et al. ∙

research

∙ 08/01/2022

Video Question Answering with Iterative Video-Text Co-Tokenization

Video question answering is a challenging task that requires understandi...

0 AJ Piergiovanni, et al. ∙

research

∙ 07/01/2022

Video + CLIP Baseline for Ego4D Long-term Action Anticipation

In this report, we introduce our adaptation of image-text models for lon...

0 Srijan Das, et al. ∙

research

∙ 06/23/2022

Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space

Humans are remarkably flexible in understanding viewpoint changes due to...

0 Jinghuan Shang, et al. ∙

research

∙ 12/07/2021

STC-mix: Space, Time, Channel mixing for Self-supervised Video Representation

Contrastive representation learning of videos highly relies on the avail...

0 Srijan Das, et al. ∙

research

∙ 12/07/2021

ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints

Learning self-supervised video representation predominantly focuses on d...

0 Srijan Das, et al. ∙

research

∙ 12/07/2021

MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection

Action detection is an essential and challenging task, especially for de...

5 Rui Dai, et al. ∙

research

∙ 11/26/2021

SWAT: Spatial Structure Within and Among Tokens

Modeling visual data as tokens (i.e., image patches), and applying atten...

1 Kumara Kahatapitiya, et al. ∙

research

∙ 11/26/2021

Self-supervised Pretraining with Classification Labels for Temporal Activity Detection

Temporal Activity Detection aims to predict activity classes per frame, ...

13 Kumara Kahatapitiya, et al. ∙

research

∙ 10/12/2021

StARformer: Transformer with State-Action-Reward Representations

Reinforcement Learning (RL) can be considered as a sequence modeling tas...

0 Jinghuan Shang, et al. ∙

research

∙ 10/08/2021

Hybrid Random Features

We propose a new class of random feature methods for linearizing softmax...

0 Krzysztof Choromanski, et al. ∙

research

∙ 09/02/2021

4D-Net for Learned Multi-Modal Alignment

We present 4D-Net, a 3D object detection approach, which utilizes 3D Poi...

7 AJ Piergiovanni, et al. ∙

research

∙ 08/02/2021

Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning

Humans learn to imitate by observing others. However, robot imitation le...

0 Jinghuan Shang, et al. ∙

research

∙ 06/28/2021

Unsupervised Discovery of Actions in Instructional Videos

In this paper we address the problem of automatically discovering atomic...

10 AJ Piergiovanni, et al. ∙

research

∙ 06/21/2021

TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?

In this paper, we introduce a novel visual representation learning which...

0 Michael S. Ryoo, et al. ∙

research

∙ 06/07/2021

Unsupervised Action Segmentation for Instructional Videos

In this paper we address the problem of automatically discovering atomic...

0 AJ Piergiovanni, et al. ∙

research

∙ 03/30/2021

Recognizing Actions in Videos from Unseen Viewpoints

Standard methods for video recognition use large CNNs designed to captur...

0 AJ Piergiovanni, et al. ∙

research

∙ 03/26/2021

Visionary: Vision architecture discovery for robot learning

We propose a vision-based architecture search algorithm for robot manipu...

0 Iretiayo Akinola, et al. ∙

research

∙ 03/01/2021

Coarse-Fine Networks for Temporal Activity Detection in Videos

In this paper, we introduce 'Coarse-Fine Networks', a two-stream archite...

14 Kumara Kahatapitiya, et al. ∙

research

∙ 11/13/2020

Reducing Inference Latency with Concurrent Architectures for Image Recognition

Satisfying the high computation demand of modern deep learning architect...

0 Ramyad Hadidi, et al. ∙

research

∙ 08/18/2020

AssembleNet++: Assembling Modality Representations via Attention Connections

We create a family of powerful video models which are able to: (i) learn...

0 Michael S. Ryoo, et al. ∙

research

∙ 08/11/2020

Adversarial Generative Grammars for Human Activity Prediction

In this paper we propose an adversarial generative grammar model for fut...

15 AJ Piergiovanni, et al. ∙

research

∙ 07/23/2020

AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification

Convolutional operations have two limitations: (1) do not explicitly mod...

10 Xiaofang Wang, et al. ∙

research

∙ 07/10/2020

AViD Dataset: Anonymized Videos from Diverse Countries

We introduce a new public video dataset for action recognition: Anonymiz...

0 AJ Piergiovanni, et al. ∙

research

∙ 03/13/2020

Edge-Tailored Perception: Fast Inferencing in-the-Edge with Efficient Model Distribution

The rise of deep neural networks (DNNs) is inspiring new studies in myri...

0 Ramyad Hadidi, et al. ∙

research

∙ 02/26/2020

Evolving Losses for Unsupervised Video Representation Learning

We present a new method to learn video representations from large-scale ...

0 AJ Piergiovanni, et al. ∙

research

∙ 11/26/2019

Password-conditioned Anonymization and Deanonymization with Face Identity Transformers

Cameras are prevalent in our daily lives, and enable many useful systems...

3 Xiuye Gu, et al. ∙

research

∙ 10/15/2019

Tiny Video Networks

Video understanding is a challenging problem with great impact on the ab...

0 AJ Piergiovanni, et al. ∙

research

∙ 10/08/2019

Model-based Behavioral Cloning with Future Image Similarity Learning

We present a visual imitation learning framework that enables learning o...

9 Alan Wu, et al. ∙

research

∙ 06/07/2019

Evolving Losses for Unlabeled Video Representation Learning

We present a new method to learn video representations from unlabeled da...

0 AJ Piergiovanni, et al. ∙

research

∙ 05/30/2019

AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures

Learning to represent videos is a very challenging task both algorithmic...

0 Michael S. Ryoo, et al. ∙

research

∙ 04/18/2019

Early Detection of Injuries in MLB Pitchers from Video

Injuries are a major cost in sports. Teams spend millions of dollars eve...

0 AJ Piergiovanni, et al. ∙

research

∙ 02/01/2019

Learning Differentiable Grammars for Continuous Data

This paper proposes a novel algorithm which learns a formal regular gram...

0 AJ Piergiovanni, et al. ∙

research

∙ 11/26/2018

Evolving Space-Time Neural Architectures for Videos

In this paper, we present a new method for evolving video CNN models to ...

2 AJ Piergiovanni, et al. ∙

research

∙ 10/02/2018

Representation Flow for Action Recognition

In this paper, we propose a convolutional layer inspired by optical flow...

0 AJ Piergiovanni, et al. ∙

research

∙ 06/21/2018

Learning Shared Multimodal Embeddings with Unpaired Data

In this paper, we propose a method to learn a joint multimodal embedding...

0 AJ Piergiovanni, et al. ∙

research

∙ 05/20/2018

Learning Real-World Robot Policies by Dreaming

Learning to control robots directly based on images is a primary challen...

0 AJ Piergiovanni, et al. ∙

research

∙ 04/09/2018

Fine-grained Activity Recognition in Baseball Videos

In this paper, we introduce a challenging new dataset, MLB-YouTube, desi...

0 AJ Piergiovanni, et al. ∙

research

∙ 03/30/2018

Learning to Anonymize Faces for Privacy Preserving Action Detection

There is an increasing concern in computer vision devices invading the p...

0 Zhongzheng Ren, et al. ∙

research

∙ 03/29/2018

Joint Person Segmentation and Identification in Synchronized First- and Third-person Videos

In a world in which cameras are becoming more and more pervasive, scenes...

0 Mingze Xu, et al. ∙

research

∙ 03/16/2018

Activity Detection with Latent Sub-event Hierarchy Learning

In this paper, we introduce a new convolutional layer named the Temporal...

0 AJ Piergiovanni, et al. ∙

research

∙ 12/05/2017

Learning Latent Super-Events to Detect Multiple Activities in Videos

In this paper, we introduce the concept of learning latent super-events ...

0 AJ Piergiovanni, et al. ∙

research

∙ 08/03/2017

Extreme Low Resolution Activity Recognition with Multi-Siamese Embedding Learning

This paper presents an approach for recognition of human activities from...

0 Michael S. Ryoo, et al. ∙

research

∙ 05/20/2017

Forecasting Hands and Objects in Future Frames

This paper presents an approach to forecast future presence and location...

0 Chenyou Fan, et al. ∙

research

∙ 03/03/2017

Learning Robot Activities from First-Person Human Videos Using Convolutional Future Regression

We design a new approach that allows robot learning of new activities fr...

0 Jangwon Lee, et al. ∙

research

∙ 03/01/2017

Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions

In this paper, we present a general framework for learning social afford...

0 Tianmin Shu, et al. ∙

Michael S. Ryoo

Featured Co-authors