RoI Tanh-polar Transformer Network for Face Parsing in the Wild

by   Yiming Lin, et al.

Face parsing aims to predict pixel-wise labels for facial components of a target face in an image. Existing approaches usually crop the target face from the input image with respect to a bounding box calculated during pre-processing, and thus can only parse inner facial Regions of Interest (RoIs). Peripheral regions like hair are ignored and nearby faces that are partially included in the bounding box can cause distractions. Moreover, these methods are only trained and evaluated on near-frontal portrait images and thus their performance for in-the-wild cases were unexplored. To address these issues, this paper makes three contributions. First, we introduce iBugMask dataset for face parsing in the wild containing 1,000 manually annotated images with large variations in sizes, poses, expressions and background, and Helen-LP, a large-pose training set containing 21,866 images generated using head pose augmentation. Second, we propose RoI Tanh-polar transform that warps the whole image to a Tanh-polar representation with a fixed ratio between the face area and the context, guided by the target bounding box. The new representation contains all information in the original image, and allows for rotation equivariance in the convolutional neural networks (CNNs). Third, we propose a hybrid residual representation learning block, coined HybridBlock, that contains convolutional layers in both the Tanh-polar space and the Tanh-Cartesian space, allowing for receptive fields of different shapes in CNNs. Through extensive experiments, we show that the proposed method significantly improves the state-of-the-art for face parsing in the wild.



There are no comments yet.


page 2

page 4

page 7

page 8

page 9

page 12


Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild

We present a framework for robust face detection and landmark localisati...

Interlinked Convolutional Neural Networks for Face Parsing

Face parsing is a basic task in face image analysis. It amounts to label...

Face Parsing with RoI Tanh-Warping

Face parsing computes pixel-wise label maps for different semantic compo...

img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation

We propose real-time, six degrees of freedom (6DoF), 3D face pose estima...

Improving Head Pose Estimation with a Combined Loss and Bounding Box Margin Adjustment

We address a problem of estimating pose of a person's head from its RGB ...

Convolutional Point-set Representation: A Convolutional Bridge Between a Densely Annotated Image and 3D Face Alignment

We present a robust method for estimating the facial pose and shape info...

Face Verification in Polar Frequency Domain: a Biologically Motivated Approach

We present a novel local-based face verification system whose components...

Code Repositories


Official Pytorch implementation of 'RoI Tanh-polar Transformer Network for Face Parsing in the Wild.'

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.