Synthesizing Images of Humans in Unseen Poses

04/20/2018
by   Guha Balakrishnan, et al.
1

We address the computational problem of novel human pose synthesis. Given an image of a person and a desired pose, we produce a depiction of that person in that pose, retaining the appearance of both the person and background. We present a modular generative neural network that synthesizes unseen poses using training pairs of images and poses taken from human action videos. Our network separates a scene into different body part and background layers, moves body parts to new locations and refines their appearances, and composites the new foreground with a hole-filled background. These subtasks, implemented with separate modules, are trained jointly using only a single target image as a supervised label. We use an adversarial discriminator to force our network to synthesize realistic details conditioned on pose. We demonstrate image synthesis results on three action classes: golf, yoga/workouts and tennis, and show that our method produces accurate results within action classes as well as across action classes. Given a sequence of desired poses, we also produce coherent videos of actions.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 7

page 8

research
12/07/2020

Pose-Guided Human Animation from a Single Image in the Wild

We present a new pose transfer method for synthesizing a human animation...
research
09/27/2018

Unsupervised Person Image Synthesis in Arbitrary Poses

We present a novel approach for synthesizing photo-realistic images of p...
research
06/30/2023

DisCo: Disentangled Control for Referring Human Dance Generation in Real World

Generative AI has made significant strides in computer vision, particula...
research
08/01/2022

Exploring the GLIDE model for Human Action-effect Prediction

We address the following action-effect prediction task. Given an image d...
research
04/01/2017

Customizing First Person Image Through Desired Actions

This paper studies a problem of inverse visual path planning: creating a...
research
11/25/2018

PCGAN: Partition-Controlled Human Image Generation

Human image generation is a very challenging task since it is affected b...
research
04/17/2019

Vid2Game: Controllable Characters Extracted from Real-World Videos

We are given a video of a person performing a certain activity, from whi...

Please sign up or login with your details

Forgot password? Click here to reset