research
∙
09/21/2023
AceGPT, Localizing Large Language Models in Arabic
This paper explores the imperative need and methodology for developing a...
share
research
∙
07/09/2023
Restricted Generative Projection for One-Class Classification and Anomaly Detection
We present a simple framework for one-class classification and anomaly d...
research
∙
04/06/2023
NTK-SAP: Improving neural network pruning by aligning training dynamics
Pruning neural networks before training has received increasing interest...
research
∙
02/28/2023
Double Dynamic Sparse Training for GANs
The past decade has witnessed a drastic increase in modern deep neural n...
research
∙
02/27/2023
Invariant Layers for Graphs with Nodes of Different Types
Neural networks that satisfy invariance with respect to input permutatio...
research
∙
11/27/2022
Adversarial Rademacher Complexity of Deep Neural Networks
Deep neural networks are vulnerable to adversarial attacks. Ideally, a r...
research
∙
11/27/2022
DigGAN: Discriminator gradIent Gap Regularization for GAN Training with Limited Data
Generative adversarial nets (GANs) have been remarkably successful at le...
research
∙
10/21/2022
When Expressivity Meets Trainability: Fewer than n Neurons Can Work
Modern neural networks are often quite wide, causing large memory and co...
research
∙
10/03/2022
Stability Analysis and Generalization Bounds of Adversarial Training
In adversarial machine learning, deep neural networks can fit the advers...
research
∙
08/20/2022
Adam Can Converge Without Any Modification on Update Rules
Ever since Reddi et al. 2018 pointed out the divergence issue of Adam, m...
research
∙
03/17/2022
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning
Model-agnostic meta-learning (MAML) and its variants have become popular...
research
∙
11/27/2021
Towards Understanding the Impact of Model Size on Differential Private Classification
Differential privacy (DP) is an essential technique for privacy-preservi...
research
∙
10/29/2021
Federated Semi-Supervised Learning with Class Distribution Mismatch
Many existing federated learning (FL) algorithms are designed for superv...
research
∙
10/08/2021
Momentum Doesn't Change the Implicit Bias
The momentum acceleration technique is widely adopted in many optimizati...
research
∙
04/24/2021
Achieving Small Test Error in Mildly Overparameterized Neural Networks
Recent theoretical works on over-parameterized neural nets have focused ...
research
∙
01/01/2021
On a Faster R-Linear Convergence Rate of the Barzilai-Borwein Method
The Barzilai-Borwein (BB) method has demonstrated great empirical succes...
research
∙
11/10/2020
Towards a Better Global Loss Landscape of GANs
Understanding of GAN training is still very limited. One major challenge...
research
∙
11/05/2020
Center-wise Local Image Mixture For Contrastive Representation Learning
Recent advances in unsupervised representation learning have experienced...
research
∙
10/29/2020
A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems
Nonconvex-concave min-max problem arises in many machine learning applic...
research
∙
09/16/2020
Landscape of Sparse Linear Network: A Brief Investigation
Network pruning, or sparse network has a long history and practical sign...
research
∙
07/02/2020
The Global Landscape of Neural Networks: An Overview
One of the major concerns for neural network training is that the non-co...
research
∙
06/25/2020
Global Convergence and Induced Kernels of Gradient-Based Meta-Learning with Neural Nets
Gradient-based meta-learning (GBML) with deep neural nets (DNNs) has bec...
research
∙
06/23/2020
Distilling Object Detectors with Task Adaptive Regularization
Current state-of-the-art object detectors are at the expense of high com...
research
∙
06/19/2020
DEED: A General Quantization Scheme for Communication Efficiency in Bits
In distributed optimization, a popular technique to reduce communication...
research
∙
12/31/2019
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity
Traditional landscape analysis of deep neural networks aims to show that...
research
∙
12/19/2019
Optimization for deep learning: theory and algorithms
When and why can a neural network be successfully trained? This article ...
research
∙
11/04/2019
Sub-Optimal Local Minima Exist for Almost All Over-parameterized Neural Networks
Does over-parameterization eliminate sub-optimal local minima for neural...
research
∙
10/10/2019
Understanding Limitation of Two Symmetrized Orders by Worst-case Complexity
It was recently found that the standard version of multi-block cyclic AD...
research
∙
09/16/2019
Off-road Autonomous Vehicles Traversability Analysis and Trajectory Planning Based on Deep Inverse Reinforcement Learning
Terrain traversability analysis is a fundamental issue to achieve the au...
research
∙
04/11/2019
Max-Sliced Wasserstein Distance and its use for GANs
Generative adversarial nets (GANs) and variational auto-encoders have si...
research
∙
12/28/2018
Over-Parameterized Deep Neural Networks Have No Strict Local Minima For Any Continuous Activations
In this paper, we study the loss surface of the over-parameterized fully...
research
∙
08/08/2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
This paper studies a class of adaptive gradient based momentum algorithm...
research
∙
05/22/2018
Adding One Neuron Can Eliminate All Bad Local Minima
One of the main difficulties in analyzing neural networks is the non-con...
research
∙
02/19/2018
Understanding the Loss Surface of Neural Networks for Binary Classification
It is widely conjectured that the reason that training algorithms for ne...
research
∙
02/15/2017
Training Language Models Using Target-Propagation
While Truncated Back-Propagation through Time (BPTT) is the most popular...
research
∙
12/15/2015
