Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation

10/07/2019
by   Hang Gao, et al.
21

Convolutional networks are not aware of an object's geometric variations, which leads to inefficient utilization of model and data capacity. To overcome this issue, recent works on deformation modeling seek to spatially reconfigure the data towards a common arrangement such that semantic recognition suffers less from deformation. This is typically done by augmenting static operators with learned free-form sampling grids in the image space, dynamically tuned to the data and task for adapting the receptive field. Yet adapting the receptive field does not quite reach the actual goal – what really matters to the network is the "effective" receptive field (ERF), which reflects how much each pixel contributes. It is thus natural to design other approaches to adapt the ERF directly during runtime. In this work, we instantiate one possible solution as Deformable Kernels (DKs), a family of novel and generic convolutional operators for handling object deformations by directly adapting the ERF while leaving the receptive field untouched. At the heart of our method is the ability to resample the original kernel space towards recovering the deformation of objects. This approach is justified with theoretical insights that the ERF is strictly determined by data sampling locations and kernel values. We implement DKs as generic drop-in replacements of rigid kernels and conduct a series of empirical studies whose results conform with our theories. Over several tasks and standard base models, our approach compares favorably against prior works that adapt during runtime. In addition, further experiments suggest a working mechanism orthogonal and complementary to previous works.

READ FULL TEXT

page 2

page 6

page 10

research
06/09/2020

An Efficient Accelerator Design Methodology for Deformable Convolutional Networks

Deformable convolutional networks have demonstrated outstanding performa...
research
04/25/2019

Blurring the Line Between Structure and Learning to Optimize and Adapt Receptive Fields

The visual world is vast and varied, but its variations divide into stru...
research
11/23/2022

Rega-Net:Retina Gabor Attention for Deep Convolutional Neural Networks

Extensive research works demonstrate that the attention mechanism in con...
research
07/18/2020

Malleable 2.5D Convolution: Learning Receptive Fields along the Depth-axis for RGB-D Scene Parsing

Depth data provide geometric information that can bring progress in RGB-...
research
06/17/2014

Replicating Kernels with a Short Stride Allows Sparse Reconstructions with Fewer Independent Kernels

In sparse coding it is common to tile an image into nonoverlapping patch...
research
05/06/2019

CARAFE: Content-Aware ReAssembly of FEatures

Feature upsampling is a key operation in a number of modern convolutiona...

Please sign up or login with your details

Forgot password? Click here to reset