Log In Sign Up

Rethinking the Zigzag Flattening for Image Reading

by   Qingsong Zhao, et al.

Sequence ordering of word vector matters a lot to text reading, which has been proven in natural language processing (NLP). However, the rule of different sequence ordering in computer vision (CV) was not well explored, e.g., why the "zigzag" flattening (ZF) is commonly utilized as a default option to get the image patches ordering in vision transformers (ViTs). Notably, when decomposing multi-scale images, the ZF could not maintain the invariance of feature point positions. To this end, we investigate the Hilbert fractal flattening (HF) as another method for sequence ordering in CV and contrast it against ZF. The HF has proven to be superior to other curves in maintaining spatial locality, when performing multi-scale transformations of dimensional space. And it can be easily plugged into most deep neural networks (DNNs). Extensive experiments demonstrate that it can yield consistent and significant performance boosts for a variety of architectures. Finally, we hope that our studies spark further research about the flattening strategy of image reading.


page 1

page 3

page 6


Vicinity Vision Transformer

Vision transformers have shown great success on numerous computer vision...

An Attention Score Based Attacker for Black-box NLP Classifier

Deep neural networks have a wide range of applications in solving variou...

RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

In fine-grained image recognition (FGIR), the localization and amplifica...

Neural Networks as Explicit Word-Based Rules

Filters of convolutional networks used in computer vision are often visu...

Representing Whole Slide Cancer Image Features with Hilbert Curves

Regions of Interest (ROI) contain morphological features in pathology wh...

HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction

In accelerated MRI reconstruction, the anatomy of a patient is recovered...

Exploring Multi-Scale Feature Propagation and Communication for Image Super Resolution

Multi-scale techniques have achieved great success in a wide range of co...