Customizing First Person Image Through Desired Actions

by   Shan Su, et al.

This paper studies a problem of inverse visual path planning: creating a visual scene from a first person action. Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible. As a proof-of-concept, we focus on linking visual experiences induced by walking. A key innovation of this paper is a concept of ActionTunnel---a 3D virtual tunnel along the future trajectory encoding what the wearer will visually experience as moving into the scene. This connects two distinctive first person images through similar walking paths. Our method takes a first person image with a user defined future trajectory and outputs a new image that can afford the future motion. The image is created by combining present and future ActionTunnels in 3D where the missing pixels in adjoining area are computed by a generative adversarial network. Our work can provide a travel across different first person experiences in diverse real world scenes.



There are no comments yet.


page 1

page 3

page 4

page 5

page 6

page 7

page 8


Synthesizing Images of Humans in Unseen Poses

We address the computational problem of novel human pose synthesis. Give...

VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss

Generating a virtual try-on image from in-shop clothing images and a mod...

Predicting 3D Human Dynamics from Video

Given a video of a person in action, we can easily guess the 3D future m...

Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image

In this paper, we propose a deep convolutional recurrent neural network ...

Future Localization from an Egocentric Depth Image

This paper presents a method for future localization: to predict a set o...

Stochastic Scene-Aware Motion Prediction

A long-standing goal in computer vision is to capture, model, and realis...

TITAN: Future Forecast using Action Priors

We consider the problem of predicting the future trajectory of scene age...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.