DeepAI AI Chat
Log In Sign Up

Dual Convexified Convolutional Neural Networks

by   Site Bai, et al.
Purdue University

We propose the framework of dual convexified convolutional neural networks (DCCNNs). In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs), and then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates. Our approach reduces the memory overhead of constructing a large kernel matrix and eliminates the ambiguity of factorizing the matrix. Due to the low-rank structure in CCNNs and the related subdifferential of nuclear norms, there is no closed-form expression to recover the primal solution from the dual solution. To overcome this, we propose a highly novel weight recovery algorithm, which takes the dual solution and the kernel information as the input, and recovers the linear and convolutional weights of a CCNN. Furthermore, our recovery algorithm exploits the low-rank structure and imposes a small number of filters indirectly, which reduces the parameter size. As a result, DCCNNs inherit all the statistical benefits of CCNNs, while enjoying a more formal and efficient workflow.


page 1

page 2

page 3

page 4


Bundle Method Sketching for Low Rank Semidefinite Programming

In this paper, we show that the bundle method can be applied to solve se...

Primal-Dual Block Frank-Wolfe

We propose a variant of the Frank-Wolfe algorithm for solving a class of...

Graph Coloring and Semidefinite Rank

This paper considers the interplay between semidefinite programming, mat...

Bayesian Inference of Random Dot Product Graphs via Conic Programming

We present a convex cone program to infer the latent probability matrix ...

Sparse Algorithm for Robust LSSVM in Primal Space

As enjoying the closed form solution, least squares support vector machi...

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Reconstruction of signals from undersampled and noisy measurements is a ...

Understanding the Covariance Structure of Convolutional Filters

Neural network weights are typically initialized at random from univaria...