DeepAI
Log In Sign Up

An Efficient Accelerator Design Methodology for Deformable Convolutional Networks

06/09/2020
by   Saehyun Ahn, et al.
0

Deformable convolutional networks have demonstrated outstanding performance in object recognition tasks with an effective feature extraction. Unlike standard convolution, the deformable convolution decides the receptive field size using dynamically generated offsets, which leads to an irregular memory access. Especially, the memory access pattern varies both spatially and temporally, making static optimization ineffective. Thus, a naive implementation would lead to an excessive memory footprint. In this paper, we present a novel approach to accelerate deformable convolution on FPGA. First, we propose a novel training method to reduce the size of the receptive field in the deformable convolutional layer without compromising accuracy. By optimizing the receptive field, we can compress the maximum size of the receptive field by 12.6 times. Second, we propose an efficient systolic architecture to maximize its efficiency. We then implement our design on FPGA to support the optimized dataflow. Experimental results show that our accelerator achieves up to 17.25 times speedup over the state-of-the-art accelerator.

READ FULL TEXT
06/12/2020

CoDeNet: Algorithm-hardware Co-design for Deformable Convolution

Deploying deep learning models on embedded systems for computer vision t...
07/06/2021

Energy-Efficient Accelerator Design for Deformable Convolution Networks

Deformable convolution networks (DCNs) proposed to address the image rec...
10/07/2019

Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation

Convolutional networks are not aware of an object's geometric variations...
02/19/2020

Algorithm-hardware Co-design for Deformable Convolution

FPGAs provide a flexible and efficient platform to accelerate rapidly-ch...
09/11/2021

In-filter Computing For Designing Ultra-light Acoustic Pattern Recognizers

We present a novel in-filter computing framework that can be used for de...
10/27/2022

Deformable Temporal Convolutional Networks for Monaural Noisy Reverberant Speech Separation

Speech separation models are used for isolating individual speakers in m...
06/16/2022

An Improved Normed-Deformable Convolution for Crowd Counting

In recent years, crowd counting has become an important issue in compute...