Uformer-ICS: A Specialized U-Shaped Transformer for Image Compressive Sensing

09/05/2022
by   Kuiyuan Zhang, et al.
2

Recently, several studies have applied deep convolutional neural networks (CNNs) in image compressive sensing (CS) tasks to improve reconstruction quality. However, convolutional layers generally have a small receptive field; therefore, capturing long-range pixel correlations using CNNs is challenging, which limits their reconstruction performance in image CS tasks. Considering this limitation, we propose a U-shaped transformer for image CS tasks, called the Uformer-ICS. We develop a projection-based transformer block by integrating the prior projection knowledge of CS into the original transformer blocks, and then build a symmetrical reconstruction model using the projection-based transformer blocks and residual convolutional blocks. Compared with previous CNN-based CS methods that can only exploit local image features, the proposed reconstruction model can simultaneously utilize the local features and long-range dependencies of an image, and the prior projection knowledge of the CS theory. Additionally, we design an adaptive sampling model that can adaptively sample image blocks based on block sparsity, which can ensure that the compressed results retain the maximum possible information of the original image under a fixed sampling ratio. The proposed Uformer-ICS is an end-to-end framework that simultaneously learns the sampling and reconstruction processes. Experimental results demonstrate that it achieves significantly better reconstruction performance than existing state-of-the-art deep learning-based CS methods.

READ FULL TEXT

page 1

page 5

page 6

page 10

page 12

research
01/31/2018

ConvCSNet: A Convolutional Compressive Sensing Framework Based on Deep Learning

Compressive sensing (CS), aiming to reconstruct an image/signal from a s...
research
03/21/2022

Adaptive and Cascaded Compressive Sensing

Scene-dependent adaptive compressive sensing (CS) has been a long pursui...
research
08/28/2019

Multi-Channel Deep Networks for Block-Based Image Compressive Sensing

Incorporating deep neural networks in image compressive sensing (CS) rec...
research
12/31/2021

CSformer: Bridging Convolution and Transformer for Compressive Sensing

Convolution neural networks (CNNs) have succeeded in compressive image s...
research
07/25/2022

TransCL: Transformer Makes Strong and Flexible Compressive Learning

Compressive learning (CL) is an emerging framework that integrates signa...
research
11/12/2021

Transformer-based Image Compression

A Transformer-based Image Compression (TIC) approach is developed which ...

Please sign up or login with your details

Forgot password? Click here to reset