DeepAI AI Chat
Log In Sign Up

Dual Convexified Convolutional Neural Networks

05/27/2022
by   Site Bai, et al.
Purdue University
0

We propose the framework of dual convexified convolutional neural networks (DCCNNs). In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs), and then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates. Our approach reduces the memory overhead of constructing a large kernel matrix and eliminates the ambiguity of factorizing the matrix. Due to the low-rank structure in CCNNs and the related subdifferential of nuclear norms, there is no closed-form expression to recover the primal solution from the dual solution. To overcome this, we propose a highly novel weight recovery algorithm, which takes the dual solution and the kernel information as the input, and recovers the linear and convolutional weights of a CCNN. Furthermore, our recovery algorithm exploits the low-rank structure and imposes a small number of filters indirectly, which reduces the parameter size. As a result, DCCNNs inherit all the statistical benefits of CCNNs, while enjoying a more formal and efficient workflow.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/11/2019

Bundle Method Sketching for Low Rank Semidefinite Programming

In this paper, we show that the bundle method can be applied to solve se...
06/06/2019

Primal-Dual Block Frank-Wolfe

We propose a variant of the Frank-Wolfe algorithm for solving a class of...
02/21/2022

Graph Coloring and Semidefinite Rank

This paper considers the interplay between semidefinite programming, mat...
01/06/2021

Bayesian Inference of Random Dot Product Graphs via Conic Programming

We present a convex cone program to infer the latent probability matrix ...
02/07/2017

Sparse Algorithm for Robust LSSVM in Primal Space

As enjoying the closed form solution, least squares support vector machi...
10/24/2021

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Reconstruction of signals from undersampled and noisy measurements is a ...
10/07/2022

Understanding the Covariance Structure of Convolutional Filters

Neural network weights are typically initialized at random from univaria...