Dual Convexified Convolutional Neural Networks

05/27/2022
by   Site Bai, et al.
0

We propose the framework of dual convexified convolutional neural networks (DCCNNs). In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs), and then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates. Our approach reduces the memory overhead of constructing a large kernel matrix and eliminates the ambiguity of factorizing the matrix. Due to the low-rank structure in CCNNs and the related subdifferential of nuclear norms, there is no closed-form expression to recover the primal solution from the dual solution. To overcome this, we propose a highly novel weight recovery algorithm, which takes the dual solution and the kernel information as the input, and recovers the linear and convolutional weights of a CCNN. Furthermore, our recovery algorithm exploits the low-rank structure and imposes a small number of filters indirectly, which reduces the parameter size. As a result, DCCNNs inherit all the statistical benefits of CCNNs, while enjoying a more formal and efficient workflow.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/11/2019

Bundle Method Sketching for Low Rank Semidefinite Programming

In this paper, we show that the bundle method can be applied to solve se...
research
06/06/2019

Primal-Dual Block Frank-Wolfe

We propose a variant of the Frank-Wolfe algorithm for solving a class of...
research
02/21/2022

Graph Coloring and Semidefinite Rank

This paper considers the interplay between semidefinite programming, mat...
research
01/06/2021

Bayesian Inference of Random Dot Product Graphs via Conic Programming

We present a convex cone program to infer the latent probability matrix ...
research
02/07/2017

Sparse Algorithm for Robust LSSVM in Primal Space

As enjoying the closed form solution, least squares support vector machi...
research
10/24/2021

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Reconstruction of signals from undersampled and noisy measurements is a ...
research
10/07/2022

Understanding the Covariance Structure of Convolutional Filters

Neural network weights are typically initialized at random from univaria...

Please sign up or login with your details

Forgot password? Click here to reset