Aggregated Pyramid Vision Transformer: Split-transform-merge Strategy for Image Recognition without Convolutions

by   Rui-Yang Ju, et al.

With the achievements of Transformer in the field of natural language processing, the encoder-decoder and the attention mechanism in Transformer have been applied to computer vision. Recently, in multiple tasks of computer vision (image classification, object detection, semantic segmentation, etc.), state-of-the-art convolutional neural networks have introduced some concepts of Transformer. This proves that Transformer has a good prospect in the field of image recognition. After Vision Transformer was proposed, more and more works began to use self-attention to completely replace the convolutional layer. This work is based on Vision Transformer, combined with the pyramid architecture, using Split-transform-merge to propose the group encoder and name the network architecture Aggregated Pyramid Vision Transformer (APVT). We perform image classification tasks on the CIFAR-10 dataset and object detection tasks on the COCO 2017 dataset. Compared with other network architectures that use Transformer as the backbone, APVT has excellent results while reducing the computational cost. We hope this improved strategy can provide a reference for future Transformer research in computer vision.



page 1

page 2

page 3

page 4


An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

While the Transformer architecture has become the de-facto standard for ...

Vision Transformer with Convolutions Architecture Search

Transformers exhibit great advantages in handling computer vision tasks....

CvT-ASSD: Convolutional vision-Transformer Based Attentive Single Shot MultiBox Detector

Due to the success of Bidirectional Encoder Representations from Transfo...

Towards End-to-End Image Compression and Analysis with Transformers

We propose an end-to-end image compression and analysis model with Trans...

Visual Concept Reasoning Networks

A split-transform-merge strategy has been broadly used as an architectur...

Transformer-Unet: Raw Image Processing with Unet

Medical image segmentation have drawn massive attention as it is importa...

Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding

This paper presents a new Vision Transformer (ViT) architecture Multi-Sc...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.