Plan-Recognition-Driven Attention Modeling for Visual Recognition

12/02/2018
by   Yantian Zha, et al.
0

Human visual recognition of activities or external agents involves an interplay between high-level plan recognition and low-level perception. Given that, a natural question to ask is: can low-level perception be improved by high-level plan recognition? We formulate the problem of leveraging recognized plans to generate better top-down attention maps gazzaniga2009,baluch2011 to improve the perception performance. We call these top-down attention maps specifically as plan-recognition-driven attention maps. To address this problem, we introduce the Pixel Dynamics Network. Pixel Dynamics Network serves as an observation model, which predicts next states of object points at each pixel location given observation of pixels and pixel-level action feature. This is like internally learning a pixel-level dynamics model. Pixel Dynamics Network is a kind of Convolutional Neural Network (ConvNet), with specially-designed architecture. Therefore, Pixel Dynamics Network could take the advantage of parallel computation of ConvNets, while learning the pixel-level dynamics model. We further prove the equivalence between Pixel Dynamics Network as an observation model, and the belief update in partially observable Markov decision process (POMDP) framework. We evaluate our Pixel Dynamics Network in event recognition tasks. We build an event recognition system, ER-PRN, which takes Pixel Dynamics Network as a subroutine, to recognize events based on observations augmented by plan-recognition-driven attention.

READ FULL TEXT
research
03/06/2017

Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation

Manipulation of deformable objects, such as ropes and cloth, is an impor...
research
10/21/2021

Pixel-Level Face Image Quality Assessment for Explainable Face Recognition

An essential factor to achieve high performance in face recognition syst...
research
12/16/2019

Conditions for Hierarchical Supervisory Control under Partial Observation

The fundamental problem in hierarchical supervisory control under partia...
research
09/15/2020

Gravitational Models Explain Shifts on Human Visual Attention

Visual attention refers to the human brain's ability to select relevant ...
research
03/23/2015

Superpixelizing Binary MRF for Image Labeling Problems

Superpixels have become prevalent in computer vision. They have been use...
research
06/14/2022

Pixel-by-pixel Mean Opinion Score (pMOS) for No-Reference Image Quality Assessment

Deep-learning based techniques have contributed to the remarkable progre...
research
12/05/2017

Recognizing Plans by Learning Embeddings from Observed Action Distributions

Recent advances in visual activity recognition have raised the possibili...

Please sign up or login with your details

Forgot password? Click here to reset