Representation learning from videos in-the-wild: An object-centric approach

10/06/2020
by   Rob Romijnders, et al.
6

We propose a method to learn image representations from uncurated videos. We combine a supervised loss from off-the-shelf object detectors and self-supervised losses which naturally arise from the video-shot-frame-object hierarchy present in each video. We report competitive results on 19 transfer learning tasks of the Visual Task Adaptation Benchmark (VTAB), and on 8 out-of-distribution-generalization tasks, and discuss the benefits and shortcomings of the proposed approach. In particular, it improves over the baseline on all 18/19 few-shot learning tasks and 8/8 out-of-distribution generalization tasks. Finally, we perform several ablation studies and analyze the impact of the pretrained object detector on the performance across this suite of tasks.

READ FULL TEXT

page 1

page 3

page 13

research
12/05/2019

Self-Supervised Learning of Video-Induced Visual Invariances

We propose a general framework for self-supervised learning of transfera...
research
10/12/2022

Self-supervised video pretraining yields strong image representations

Videos contain far more information than still images and hold the poten...
research
09/05/2022

A Study on Representation Transfer for Few-Shot Learning

Few-shot classification aims to learn to classify new object categories ...
research
03/26/2021

Few-Shot Learning for Video Object Detection in a Transfer-Learning Scheme

Different from static images, videos contain additional temporal and spa...
research
03/24/2022

A Rationale-Centric Framework for Human-in-the-loop Machine Learning

We present a novel rationale-centric framework with human-in-the-loop – ...
research
02/26/2020

Evolving Losses for Unsupervised Video Representation Learning

We present a new method to learn video representations from large-scale ...
research
06/14/2021

A Self-Supervised Framework for Function Learning and Extrapolation

Understanding how agents learn to generalize – and, in particular, to ex...

Please sign up or login with your details

Forgot password? Click here to reset