Customizing First Person Image Through Desired Actions

04/01/2017
by   Shan Su, et al.
0

This paper studies a problem of inverse visual path planning: creating a visual scene from a first person action. Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible. As a proof-of-concept, we focus on linking visual experiences induced by walking. A key innovation of this paper is a concept of ActionTunnel---a 3D virtual tunnel along the future trajectory encoding what the wearer will visually experience as moving into the scene. This connects two distinctive first person images through similar walking paths. Our method takes a first person image with a user defined future trajectory and outputs a new image that can afford the future motion. The image is created by combining present and future ActionTunnels in 3D where the missing pixels in adjoining area are computed by a generative adversarial network. Our work can provide a travel across different first person experiences in diverse real world scenes.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 4

page 5

page 6

page 7

page 8

04/20/2018

Synthesizing Images of Humans in Unseen Poses

We address the computational problem of novel human pose synthesis. Give...
11/12/2019

VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss

Generating a virtual try-on image from in-shop clothing images and a mod...
08/13/2019

Predicting 3D Human Dynamics from Video

Given a video of a person in action, we can easily guess the 3D future m...
06/09/2020

Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image

In this paper, we propose a deep convolutional recurrent neural network ...
09/07/2015

Future Localization from an Egocentric Depth Image

This paper presents a method for future localization: to predict a set o...
08/18/2021

Stochastic Scene-Aware Motion Prediction

A long-standing goal in computer vision is to capture, model, and realis...
03/31/2020

TITAN: Future Forecast using Action Priors

We consider the problem of predicting the future trajectory of scene age...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.