An Attention Module for Convolutional Neural Networks

08/18/2021
by   Zhu Baozhou, et al.
0

Attention mechanism has been regarded as an advanced technique to capture long-range feature interactions and to boost the representation capability for convolutional neural networks. However, we found two ignored problems in current attentional activations-based models: the approximation problem and the insufficient capacity problem of the attention maps. To solve the two problems together, we initially propose an attention module for convolutional neural networks by developing an AW-convolution, where the shape of attention maps matches that of the weights rather than the activations. Our proposed attention module is a complementary method to previous attention-based schemes, such as those that apply the attention mechanism to explore the relationship between channel-wise and spatial features. Experiments on several datasets for image classification and object detection tasks show the effectiveness of our proposed attention module. In particular, our proposed attention module achieves 1.00 ResNet101 baseline and 0.63 COCO-style Average Precision improvement on the COCO object detection on top of a Faster R-CNN baseline with the backbone of ResNet101-FPN. When integrating with the previous attentional activations-based models, our proposed attention module can further increase their Top-1 accuracy on ImageNet classification by up to 0.57 the COCO object detection by up to 0.45. Code and pre-trained models will be publicly available.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/17/2018

CBAM: Convolutional Block Attention Module

We propose Convolutional Block Attention Module (CBAM), a simple yet eff...
research
04/13/2023

Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution

This paper proposes a novel module called middle spectrum grouped convol...
research
04/06/2023

RFAConv: Innovating Spatital Attention and Standard Convolutional Operation

Spatial attention has been demonstrated to enable convolutional neural n...
research
04/25/2019

HAR-Net: Joint Learning of Hybrid Attention for Single-stage Object Detection

Object detection has been a challenging task in computer vision. Althoug...
research
04/09/2019

Contextual Attention for Hand Detection in the Wild

We present Hand-CNN, a novel convolutional network architecture for dete...
research
10/22/2021

Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregation

This paper introduces a concept of layer aggregation to describe how inf...
research
07/17/2018

BAM: Bottleneck Attention Module

Recent advances in deep neural networks have been developed via architec...

Please sign up or login with your details

Forgot password? Click here to reset