research
∙
09/01/2023
Randomized Polar Codes for Anytime Distributed Machine Learning
We present a novel distributed computing framework that is robust to slo...
share
research
∙
08/08/2023
Iterative Sketching for Secure Coded Regression
In this work, we propose methods for speeding up linear regression distr...
research
∙
08/06/2023
Gradient Coding through Iterative Block Leverage Score Sampling
We generalize the leverage score sampling sketch for ℓ_2-subspace embedd...
research
∙
05/31/2023
Optimal Sets and Solution Paths of ReLU Networks
We develop an analytical framework to characterize the set of optimal Re...
research
∙
03/06/2023
Globally Optimal Training of Neural Networks with Threshold Activation Functions
Threshold activation functions are highly preferable in neural networks ...
research
∙
02/27/2023
Complex Clipping for Improved Generalization in Machine Learning
For many machine learning applications, a common input representation is...
research
∙
01/09/2023
Federated Coded Matrix Inversion
Federated learning (FL) is a decentralized model for training data distr...
research
∙
09/30/2022
Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery
The practice of deep learning has shown that neural networks generalize ...
research
∙
07/18/2022
GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction
Unrolled neural networks have recently achieved state-of-the-art acceler...
research
∙
07/13/2022
Secure Linear MDS Coded Matrix Inversion
A cumbersome operation in many scientific fields, is inverting large ful...
research
∙
05/17/2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers
Vision transformers using self-attention or its proposed alternatives ha...
research
∙
04/21/2022
Scale-Equivariant Unrolled Neural Networks for Data-Efficient Accelerated MRI Reconstruction
Unrolled neural networks have enabled state-of-the-art reconstruction pe...
research
∙
03/18/2022
Approximate Function Evaluation via Multi-Armed Bandits
We study the problem of estimating the value of a known smooth function ...
research
∙
03/18/2022
Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Bounds
We consider distributed optimization methods for problems where forming ...
research
∙
02/23/2022
Minimax Optimal Quantization of Linear Models: Information-Theoretic Limits and Efficient Algorithms
We consider the problem of quantizing a linear model learned from measur...
research
∙
02/02/2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions
We develop fast algorithms and robust software for convex optimization o...
research
∙
01/21/2022
Orthonormal Sketches for Secure Coded Regression
In this work, we propose a method for speeding up linear regression dist...
research
∙
01/21/2022
Using a Novel COVID-19 Calculator to Measure Positive U.S. Socio-Economic Impact of a COVID-19 Pre-Screening Solution (AI/ML)
The COVID-19 pandemic has been a scourge upon humanity, claiming the liv...
research
∙
01/05/2022
Using Deep Learning with Large Aggregated Datasets for COVID-19 Classification from Cough
The Covid-19 pandemic has been a scourge upon humanity, claiming the liv...
research
∙
10/18/2021
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks
Despite several attempts, the fundamental mechanisms behind the success ...
research
∙
10/13/2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program
We study non-convex subgradient flows for training two-layer ReLU neural...
research
∙
10/13/2021
Parallel Deep Neural Networks Have Zero Duality Gap
Training deep neural networks is a well-known highly non-convex problem....
research
∙
10/11/2021
Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs
Understanding the fundamental mechanism behind the success of deep neura...
research
∙
09/08/2021
Computational Polarization: An Information-theoretic Method for Resilient Computing
We introduce an error resilient distributed computing method based on an...
research
∙
07/15/2021
Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update
In second-order optimization, a potential bottleneck can be computing th...
research
∙
07/12/2021
Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
Generative Adversarial Networks (GANs) are commonly used for modeling co...
research
∙
05/15/2021
Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionality
We propose a randomized algorithm with quadratic convergence rate for co...
research
∙
05/04/2021
Training Quantized Neural Networks to Global Optimality via Semidefinite Programming
Neural networks (NNs) have been extremely successful across many tasks i...
research
∙
04/29/2021
Fast Convex Quadratic Optimization Solvers with Adaptive Sketching-based Preconditioners
We consider least-squares problems with quadratic regularization and pro...
research
∙
03/13/2021
Distributed Learning and Democratic Embeddings: Polynomial-Time Source Coding Schemes Can Achieve Minimax Lower Bounds for Distributed Gradient Descent under Communication Cons
In this work, we consider the distributed optimization setting where inf...
research
∙
03/02/2021
Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Batch Normalization (BN) is a commonly used technique to accelerate and ...
research
∙
01/07/2021
Neural Spectrahedra and Semidefinite Lifts: Global Convex Optimization of Polynomial Activation Neural Networks in Fully Polynomial-Time
The training of two-layer neural networks with nonlinear activation func...
research
∙
12/24/2020
Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
We describe the convex semi-infinite dual of the two-layer vector-output...
research
∙
12/13/2020
Adaptive and Oblivious Randomized Subspace Methods for High-Dimensional Optimization: Sharp Analysis and Lower Bounds
We propose novel randomized optimization methods for high-dimensional co...
research
∙
12/09/2020
Convex Regularization Behind Neural Reconstruction
Neural networks have shown tremendous potential for reconstructing high-...
research
∙
11/19/2020
Approximate Weighted CR Coded Matrix Multiplication
One of the most common, but at the same time expensive operations in lin...
research
∙
07/02/2020
Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization
In distributed second order optimization, a standard strategy is to aver...
research
∙
06/26/2020
Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time
We study training of Convolutional Neural Networks (CNNs) with ReLU acti...
research
∙
06/15/2020
Lower Bounds and a Near-Optimal Shrinkage Estimator for Least Squares using Random Projections
In this work, we consider the deterministic optimization using random pr...
research
∙
06/10/2020
All Local Minima are Global for Two-Layer ReLU Neural Networks: The Hidden Convex Optimization Landscape
We are interested in two-layer ReLU neural networks from an optimization...
research
∙
06/10/2020
Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimization
We propose a new randomized algorithm for solving L2-regularized least-s...
research
∙
05/21/2020
Global Multiclass Classification from Heterogeneous Local Models
Multiclass classification problems are most often solved by either train...
research
∙
03/05/2020
Straggler Robust Distributed Matrix Inverse Approximation
A cumbersome operation in numerical analysis and linear algebra, optimiz...
research
∙
02/25/2020
Convex Geometry and Duality of Over-parameterized Neural Networks
We develop a convex analytic framework for ReLU neural networks which el...
research
∙
02/25/2020
Separating the Effects of Batch Normalization on CNN Training Speed and Stability Using Classical Adaptive Filter Theory
Batch Normalization (BatchNorm) is commonly used in Convolutional Neural...
research
∙
02/24/2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-Layer Networks
We develop exact representations of two layer neural networks with recti...
research
∙
02/22/2020
Convex Duality of Deep Neural Networks
We study regularized deep neural networks and introduce an analytic fram...
research
∙
02/21/2020
Optimal Randomized First-Order Methods for Least-Squares Problems
We provide an exact analysis of a class of randomized algorithms for sol...
research
∙
02/16/2020
Distributed Averaging Methods for Randomized Second Order Optimization
We consider distributed optimization problems where forming the Hessian ...
research
∙
02/16/2020