Conceptual Compression via Deep Structure and Texture Synthesis

11/10/2020
by   Jianhui Chang, et al.
0

Existing compression methods typically focus on the removal of signal-level redundancies, while the potential and versatility of decomposing visual data into compact conceptual components still lack further study. To this end, we propose a novel conceptual compression framework that encodes visual data into compact structure and texture representations, then decodes in a deep synthesis fashion, aiming to achieve better visual reconstruction quality, flexible content manipulation, and potential support for various vision tasks. In particular, we propose to compress images by a dual-layered model consisting of two complementary visual features: 1) structure layer represented by structural maps and 2) texture layer characterized by low-dimensional deep representations. At the encoder side, the structural maps and texture representations are individually extracted and compressed, generating the compact, interpretable, inter-operable bitstreams. During the decoding stage, a hierarchical fusion GAN (HF-GAN) is proposed to learn the synthesis paradigm where the textures are rendered into the decoded structural maps, leading to high-quality reconstruction with remarkable visual realism. Extensive experiments on diverse images have demonstrated the superiority of our framework with lower bitrates, higher reconstruction quality, and increased versatility towards visual analysis and content manipulation tasks.

READ FULL TEXT

page 1

page 3

page 5

page 7

page 8

page 9

03/12/2021

Thousand to One: Semantic Prior Modeling for Conceptual Coding

Conceptual coding has been an emerging research topic recently, which en...
04/21/2020

Towards Analysis-friendly Face Representation with Scalable Feature and Texture Compression

It plays a fundamental role to compactly represent the visual informatio...
06/16/2021

Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective

Visual analytics have played an increasingly critical role in the Intern...
10/11/2019

Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning

Automatic generation of artistic glyph images is a challenging task that...
03/14/2019

Scalable Facial Image Compression with Deep Feature Reconstruction

In this paper, we propose a scalable image compression scheme, including...
10/18/2021

Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics

Video Coding for Machines (VCM) is committed to bridging to an extent se...
01/28/2021

Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GAN

Reconstructing seeing images from fMRI recordings is an absorbing resear...