Towards Active Vision for Action Localization with Reactive Control and Predictive Learning

11/09/2021
by   Shubham Trehan, et al.
0

Visual event perception tasks such as action localization have primarily focused on supervised learning settings under a static observer, i.e., the camera is static and cannot be controlled by an algorithm. They are often restricted by the quality, quantity, and diversity of annotated training data and do not often generalize to out-of-domain samples. In this work, we tackle the problem of active action localization where the goal is to localize an action while controlling the geometric and physical parameters of an active camera to keep the action in the field of view without training data. We formulate an energy-based mechanism that combines predictive learning and reactive control to perform active action localization without rewards, which can be sparse or non-existent in real-world environments. We perform extensive experiments in both simulated and real-world environments on two tasks - active object tracking and active action localization. We demonstrate that the proposed approach can generalize to different tasks and environments in a streaming fashion, without explicit rewards or training. We show that the proposed approach outperforms unsupervised baselines and obtains competitive performance compared to those trained with reinforcement learning.

READ FULL TEXT

page 6

page 8

research
04/29/2021

Learning Actor-centered Representations for Action Localization in Streaming Videos using Predictive Learning

Event perception tasks such as recognizing and localizing actions in str...
research
03/26/2020

Action Localization through Continual Predictive Learning

The problem of action recognition involves locating the action in the vi...
research
08/09/2022

Object Detection with Deep Reinforcement Learning

Object localization has been a crucial task in computer vision field. Me...
research
05/30/2016

Control of Memory, Active Perception, and Action in Minecraft

In this paper, we introduce a new set of reinforcement learning (RL) tas...
research
12/08/2020

Active Visual Localization in Partially Calibrated Environments

Humans can robustly localize themselves without a map after they get los...
research
11/28/2022

A Visual Active Search Framework for Geospatial Exploration

Many problems can be viewed as forms of geospatial search aided by aeria...
research
08/01/2016

Learning Transferable Policies for Monocular Reactive MAV Control

The ability to transfer knowledge gained in previous tasks into new cont...

Please sign up or login with your details

Forgot password? Click here to reset