DeepAI
Log In Sign Up

Who Let The Dogs Out? Modeling Dog Behavior From Visual Data

03/28/2018
by   Lucas Taylor, et al.
0

We introduce the task of directly modeling a visually intelligent agent. Computer vision typically focuses on solving various subtasks related to visual intelligence. We depart from this standard approach to computer vision; instead we directly model a visually intelligent agent. Our model takes visual information as input and directly predicts the actions of the agent. Toward this end we introduce DECADE, a large-scale dataset of ego-centric videos from a dog's perspective as well as her corresponding movements. Using this data we model how the dog acts and how the dog plans her movements. We show under a variety of metrics that given just visual input we can successfully model this intelligent agent in many situations. Moreover, the representation learned by our model encodes distinct information compared to representations trained on image classification, and our learned representation can generalize to other domains. In particular, we show strong results on the task of walkable surface estimation by using this dog modeling task as representation learning.

READ FULL TEXT

page 1

page 4

page 7

10/16/2020

What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions

Learning effective representations of visual data that generalize to a v...
12/08/2022

Task Bias in Vision-Language Models

Incidental supervision from language has become a popular approach for l...
12/01/2020

Strike on Stage: a percussion and media performance

This paper describes Strike on Stage, an interface and corresponding aud...
01/12/2016

Learning Subclass Representations for Visually-varied Image Classification

In this paper, we present a subclass-representation approach that predic...
12/29/2013

Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition

Systems based on bag-of-words models from image features collected at ma...
04/05/2016

The Curious Robot: Learning Visual Representations via Physical Interactions

What is the right supervisory signal to train visual representations? Cu...
05/30/2019

Does computer vision matter for action?

Computer vision produces representations of scene content. Much computer...