Two-level Group Convolution

10/11/2021
by   Youngkyu Lee, et al.
0

Group convolution has been widely used in order to reduce the computation time of convolution, which takes most of the training time of convolutional neural networks. However, it is well known that a large number of groups significantly reduce the performance of group convolution. In this paper, we propose a new convolution methodology called “two-level” group convolution that is robust with respect to the increase of the number of groups and suitable for multi-GPU parallel computation. We first observe that the group convolution can be interpreted as a one-level block Jacobi approximation of the standard convolution, which is a popular notion in the field of numerical analysis. In numerical analysis, there have been numerous studies on the two-level method that introduces an intergroup structure that resolves the performance degradation issue without disturbing parallel computation. Motivated by these, we introduce a coarse-level structure which promotes intergroup communication without being a bottleneck in the group convolution. We show that all the additional work induced by the coarse-level structure can be efficiently processed in a distributed memory system. Numerical results that verify the robustness of the proposed method with respect to the number of groups are presented. Moreover, we compare the proposed method to various approaches for group convolution in order to highlight the superiority of the proposed method in terms of execution time, memory efficiency, and performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/31/2019

Fully Learnable Group Convolution for Acceleration of Deep Neural Networks

Benefitted from its great success on many tasks, deep learning is increa...
research
08/16/2019

Differentiable Learning-to-Group Channels viaGroupable Convolutional Neural Networks

Group convolution, which divides the channels of ConvNets into groups, h...
research
08/16/2019

Differentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks

Group convolution, which divides the channels of ConvNets into groups, h...
research
07/11/2019

Efficient Semantic Scene Completion Network with Spatial Group Convolution

We introduce Spatial Group Convolution (SGC) for accelerating the comput...
research
10/29/2019

Derivation and Analysis of Fast Bilinear Algorithms for Convolution

The prevalence of convolution in applications within signal processing, ...
research
10/17/2022

A tradeoff between universality of equivariant models and learnability of symmetries

We prove an impossibility result, which in the context of function learn...
research
03/11/2016

Efficient forward propagation of time-sequences in convolutional neural networks using Deep Shifting

When a Convolutional Neural Network is used for on-the-fly evaluation of...

Please sign up or login with your details

Forgot password? Click here to reset