Imagine That! Leveraging Emergent Affordances for Tool Synthesis in Reaching Tasks

09/30/2019
by   Yizhe Wu, et al.
9

In this paper we investigate an artificial agent's ability to perform task-focused tool synthesis via imagination. Our motivation is to explore the richness of information captured by the latent space of an object-centric generative model – and how to exploit it. In particular, our approach employs activation maximisation of a task-based performance predictor to optimise the latent variable of a structured latent-space model in order to generate tool geometries appropriate for the task at hand. We evaluate our model using a novel dataset of synthetic reaching tasks inspired by the cognitive sciences and behavioural ecology. In doing so we examine the model's ability to imagine tools for increasingly complex scenario types, beyond those seen during training. Our experiments demonstrate that the synthesis process modifies emergent, task-relevant object affordances in a targeted and deliberate way: the agents often specifically modify aspects of the tools which relate to meaningful (yet implicitly learned) concepts such as a tool's length, width and configuration. Our results therefore suggest that task relevant object affordances are implicitly encoded as directions in a structured latent space shaped by experience.

READ FULL TEXT

page 8

page 16

page 17

research
04/14/2023

Symmetry and Complexity in Object-Centric Deep Active Inference Models

Humans perceive and interact with hundreds of objects every day. In doin...
research
04/17/2018

Learning Sparse Latent Representations with the Deep Copula Information Bottleneck

Deep latent variable models are powerful tools for representation learni...
research
08/10/2023

Exploring XAI for the Arts: Explaining Latent Space in Generative Music

Explainable AI has the potential to support more interactive and fluid c...
research
05/05/2021

Exploring emotional prototypes in a high dimensional TTS latent space

Recent TTS systems are able to generate prosodically varied and realisti...
research
03/14/2023

Controlling High-Dimensional Data With Sparse Input

We address the problem of human-in-the-loop control for generating highl...
research
10/26/2022

Leveraging Demonstrations with Latent Space Priors

Demonstrations provide insight into relevant state or action space regio...
research
05/26/2023

Structured Latent Variable Models for Articulated Object Interaction

In this paper, we investigate a scenario in which a robot learns a low-d...

Please sign up or login with your details

Forgot password? Click here to reset