SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network

11/23/2020
by   Yuzhi Zhao, et al.
16

Given a grayscale photograph, the colorization system estimates a visually plausible colorful image. Conventional methods often use semantics to colorize grayscale images. However, in these methods, only classification semantic information is embedded, resulting in semantic confusion and color bleeding in the final colorized image. To address these issues, we propose a fully automatic Saliency Map-guided Colorization with Generative Adversarial Network (SCGAN) framework. It jointly predicts the colorization and saliency map to minimize semantic confusion and color bleeding in the colorized image. Since the global features from pre-trained VGG-16-Gray network are embedded to the colorization encoder, the proposed SCGAN can be trained with much less data than state-of-the-art methods to achieve perceptually reasonable colorization. In addition, we propose a novel saliency map-based guidance method. Branches of the colorization decoder are used to predict the saliency map as a proxy target. Moreover, two hierarchical discriminators are utilized for the generated colorization and saliency map, respectively, in order to strengthen visual perception performance. The proposed system is evaluated on ImageNet validation set. Experimental results show that SCGAN can generate more reasonable colorized images than state-of-the-art techniques.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 8

page 10

page 11

page 12

page 13

page 14

page 15

12/31/2017

Context aware saliency map generation using semantic segmentation

Saliency map detection, as a method for detecting important regions of a...
01/01/2022

SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images

This paper introduces a new framework to predict visual attention of omn...
11/20/2020

SalSum: Saliency-based Video Summarization using Generative Adversarial Networks

The huge amount of video data produced daily by camera-based systems, su...
10/06/2020

Visualizing Color-wise Saliency of Black-Box Image Classification Models

Image classification based on machine learning is being commonly used. H...
12/21/2018

Saliency Guided Hierarchical Robust Visual Tracking

A saliency guided hierarchical visual tracking (SHT) algorithm containin...
02/01/2017

Visual Saliency Prediction Using a Mixture of Deep Neural Networks

Visual saliency models have recently begun to incorporate deep learning ...
04/20/2022

Situational Perception Guided Image Matting

Most automatic matting methods try to separate the salient foreground fr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.