research
∙
03/06/2023
Globally Optimal Training of Neural Networks with Threshold Activation Functions
Threshold activation functions are highly preferable in neural networks ...
share
research
∙
11/20/2022
Convexifying Transformers: Improving optimization and understanding of transformer networks
Understanding the fundamental mechanism behind the success of transforme...
research
∙
07/18/2022
GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction
Unrolled neural networks have recently achieved state-of-the-art acceler...
research
∙
05/17/2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers
Vision transformers using self-attention or its proposed alternatives ha...
research
∙
10/18/2021
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks
Despite several attempts, the fundamental mechanisms behind the success ...
research
∙
10/13/2021
Parallel Deep Neural Networks Have Zero Duality Gap
Training deep neural networks is a well-known highly non-convex problem....
research
∙
10/11/2021
Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs
Understanding the fundamental mechanism behind the success of deep neura...
research
∙
07/12/2021
Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
Generative Adversarial Networks (GANs) are commonly used for modeling co...
research
∙
03/02/2021
Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Batch Normalization (BN) is a commonly used technique to accelerate and ...
research
∙
12/24/2020
Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
We describe the convex semi-infinite dual of the two-layer vector-output...
research
∙
06/26/2020
Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time
We study training of Convolutional Neural Networks (CNNs) with ReLU acti...
research
∙
02/25/2020
Convex Geometry and Duality of Over-parameterized Neural Networks
We develop a convex analytic framework for ReLU neural networks which el...
research
∙
02/24/2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-Layer Networks
We develop exact representations of two layer neural networks with recti...
research
∙
02/22/2020
Convex Duality of Deep Neural Networks
We study regularized deep neural networks and introduce an analytic fram...
research
∙
10/25/2017