BRULÉ: Barycenter-Regularized Unsupervised Landmark Extraction

06/20/2020
by   Iaroslav Bespalov, et al.
0

Unsupervised retrieval of image features is vital for many computer vision tasks where the annotation is missing or scarce. In this work, we propose a new unsupervised approach to detect the landmarks in images, and we validate it on the popular task of human face key-points extraction. The method is based on the idea of auto-encoding the wanted landmarks in the latent space while discarding the non-essential information in the image and effectively preserving the interpretability. The interpretable latent space representation is achieved with the aid of a novel two-step regularization paradigm. The first regularization step evaluates transport distance from a given set of landmarks to the average value (the barycenter by Wasserstein distance). The second regularization step controls deviations from the barycenter by applying random geometric deformations synchronously to the initial image and to the encoded landmarks. During decoding, we add style features generated from the noise and reconstruct the initial image by the generative adversarial network (GAN) with transposed convolutions modulated by this style. We demonstrate the effectiveness of the approach both in unsupervised and in semi-supervised training scenarios using the 300-W and the CelebA datasets. The proposed regularization paradigm is shown to prevent overfitting, and the detection quality is shown to improve beyond the supervised outcome.

READ FULL TEXT

page 2

page 5

page 7

page 8

page 13

research
11/26/2020

Regularization with Latent Space Virtual Adversarial Training

Virtual Adversarial Training (VAT) has shown impressive results among re...
research
10/18/2022

Landmark Enforcement and Style Manipulation for Generative Morphing

Morph images threaten Facial Recognition Systems (FRS) by presenting as ...
research
04/08/2021

Generative Landmarks

We propose a general purpose approach to detect landmarks with improved ...
research
04/07/2021

Pretrained equivariant features improve unsupervised landmark discovery

Locating semantically meaningful landmark points is a crucial component ...
research
06/21/2019

A Cyclically-Trained Adversarial Network for Invariant Representation Learning

We propose a cyclically-trained adversarial network to learn mappings fr...
research
01/24/2022

Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN

The semantically disentangled latent subspace in GAN provides rich inter...

Please sign up or login with your details

Forgot password? Click here to reset