DeepAI
Log In Sign Up

Rethinking the Zigzag Flattening for Image Reading

02/21/2022
by   Qingsong Zhao, et al.
0

Sequence ordering of word vector matters a lot to text reading, which has been proven in natural language processing (NLP). However, the rule of different sequence ordering in computer vision (CV) was not well explored, e.g., why the "zigzag" flattening (ZF) is commonly utilized as a default option to get the image patches ordering in vision transformers (ViTs). Notably, when decomposing multi-scale images, the ZF could not maintain the invariance of feature point positions. To this end, we investigate the Hilbert fractal flattening (HF) as another method for sequence ordering in CV and contrast it against ZF. The HF has proven to be superior to other curves in maintaining spatial locality, when performing multi-scale transformations of dimensional space. And it can be easily plugged into most deep neural networks (DNNs). Extensive experiments demonstrate that it can yield consistent and significant performance boosts for a variety of architectures. Finally, we hope that our studies spark further research about the flattening strategy of image reading.

READ FULL TEXT

page 1

page 3

page 6

06/21/2022

Vicinity Vision Transformer

Vision transformers have shown great success on numerous computer vision...
12/22/2021

An Attention Score Based Attacker for Black-box NLP Classifier

Deep neural networks have a wide range of applications in solving variou...
07/17/2021

RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

In fine-grained image recognition (FGIR), the localization and amplifica...
07/10/2019

Neural Networks as Explicit Word-Based Rules

Filters of convolutional networks used in computer vision are often visu...
05/13/2020

Representing Whole Slide Cancer Image Features with Hilbert Curves

Regions of Interest (ROI) contain morphological features in pathology wh...
03/15/2022

HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction

In accelerated MRI reconstruction, the anatomy of a patient is recovered...
08/01/2020

Exploring Multi-Scale Feature Propagation and Communication for Image Super Resolution

Multi-scale techniques have achieved great success in a wide range of co...