
Memorybased Optimization Methods for ModelAgnostic MetaLearning
Recently, modelagnostic metalearning (MAML) has garnered tremendous at...
read it

Stability and Generalization of Stochastic Gradient Methods for Minimax Problems
Many machine learning problems can be formulated as minimax problems suc...
read it

A Simple yet Universal Strategy for Online Convex Optimization
Recently, several universal methods have been proposed for online convex...
read it

Randomized Stochastic VarianceReduced Methods for Stochastic Bilevel Optimization
In this paper, we consider nonconvex stochastic bilevel optimization (S...
read it

Stochastic Optimization of Area Under PrecisionRecall Curve for Deep Learning with Provable Convergence
Areas under ROC (AUROC) and precisionrecall curves (AUPRC) are common m...
read it

Online Convex Optimization with Continuous Switching Constraint
In many sequential decision making applications, the change of decision ...
read it

Revisiting Smoothed Online Learning
In this paper, we revisit the problem of smoothed online learning, in wh...
read it

Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity
eep UC (area under the ROC curve) aximization (DAM) has attracted much a...
read it

Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification
Deep AUC Maximization (DAM) is a paradigm for learning a deep neural net...
read it

VarianceReduced OffPolicy MemoryEfficient Policy Search
Offpolicy policy optimization is a challenging problem in reinforcement...
read it

Nearly Optimal Robust Method for Convex Compositional Problems with HeavyTailed Noise
In this paper, we propose robust stochastic algorithms for solving conve...
read it

Fast Objective and Duality Gap Convergence for Nonconvex Stronglyconcave Minmax Problems
This paper focuses on stochastic methods for solving smooth nonconvex s...
read it

CommunicationEfficient Distributed Stochastic AUC Maximization with Deep Neural Networks
In this paper, we study distributed algorithms for largescale AUC maxim...
read it

Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectives
Stochastic gradient descent (SGD) has been widely studied in the literat...
read it

Sharp Analysis of Epoch Stochastic Gradient Descent Ascent Methods for MinMax Optimization
Epoch gradient descent method (a.k.a. EpochGD) proposed by (Hazan and K...
read it

Minimizing Dynamic Regret and Adaptive Regret Simultaneously
Regret minimization is treated as the golden rule in the traditional stu...
read it

Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets
Adaptive gradient algorithms perform gradientbased updates using the hi...
read it

a simple and effective framework for pairwise deep metric learning
Deep metric learning (DML) has received much attention in deep learning ...
read it

Decentralized Parallel Algorithm for Training Generative Adversarial Nets
Generative Adversarial Networks (GANs) are powerful class of generative ...
read it

Learning with Longterm Remembering: Following the Lead of Mixed Stochastic Gradient
Current deep neural networks can achieve remarkable performance on a sin...
read it

Stochastic AUC Maximization with Deep Neural Networks
Stochastic AUC maximization has garnered an increasing interest due to b...
read it

Stochastic Optimization for Nonconvex InfProjection Problems
In this paper, we study a family of nonconvex and possibly nonsmooth i...
read it

A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints
Stochastic convex optimization problems with expectation constraints (SO...
read it

Stochastic PrimalDual Algorithms with Faster Convergence than O(1/√(T)) for Problems without Bilinear Structure
Previous studies on stochastic primaldual algorithms for solving minma...
read it

Why Does Stagewise Training Accelerate Convergence of Testing Error Over SGD?
Stagewise training strategy is commonly used for learning neural network...
read it

Stochastic Optimization for DC Functions and Nonsmooth Nonconvex Regularizers with Nonasymptotic Convergence
Difference of convex (DC) functions cover a broad family of nonconvex a...
read it

Solving WeaklyConvexWeaklyConcave SaddlePoint Problems as WeaklyMonotone Variational Inequality
In this paper, we consider firstorder algorithms for solving a class of...
read it

NonConvex MinMax Optimization: Provable Algorithms and Applications in Machine Learning
Minmax saddlepoint problems have broad applications in many tasks in m...
read it

Learning Discriminators as Energy Networks in Adversarial Learning
We propose a novel framework for structured prediction via adversarial l...
read it

A Unified Analysis of Stochastic Momentum Methods for Deep Learning
Stochastic momentum methods have been widely adopted in training deep ne...
read it

Universal Stagewise Learning for NonConvex Problems with Convergence on Averaged Solutions
Although stochastic gradient descent () method and its variants (e.g., s...
read it

Improving Sequential Determinantal Point Processes for Supervised Video Summarization
It is now much easier than ever before to produce videos. While the ubiq...
read it

How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization
The large volume of video content and high viewing frequency demand auto...
read it

EIGEN: EcologicallyInspired GENetic Approach for Neural Network Structure Searching
Designing the structure of neural networks is considered one of the most...
read it

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints
Recently, there emerged revived interests of designing automatic program...
read it

Learning with NonConvex Truncated Losses by SGD
Learning with a convex loss function has been a dominating paradigm for...
read it

Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions
Error bound conditions (EBC) are properties that characterize the growth...
read it

NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for NonConvex Optimization
Accelerated gradient (AG) methods are breakthroughs in convex optimizati...
read it

Firstorder Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time
Two classes of methods have been proposed for escaping from saddle point...
read it

Stochastic Nonconvex Optimization with Strong High Probability Secondorder Convergence
In this paper, we study stochastic nonconvex optimization with nonconv...
read it

On Noisy Negative Curvature Descent: Competing with Gradient Descent for Faster Nonconvex Optimization
The Hessianvector product has been utilized to find a secondorder stat...
read it

A Simple Analysis for Expconcave Empirical Minimization with Arbitrary Convex Regularizer
In this paper, we present a simple analysis of fast rates with high pr...
read it

SEPNets: Small and Effective Pattern Networks
While going deeper has been witnessed to improve the performance of conv...
read it

Efficient Feature Screening for LassoType Problems via Hybrid SafeStrong Rules
The lasso model has been widely used for model selection in data mining,...
read it

A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates
This paper focuses on convex constrained optimization problems, where th...
read it

Homotopy Smoothing for NonSmooth Problems with Lower Complexity than O(1/ε)
In this paper, we develop a novel homoto py smoothing (HOPS) algorithm...
read it

Accelerated Stochastic Subgradient Methods under Local Error Bound Condition
In this paper, we propose two accelerated stochastic subgradient method...
read it

Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient
This work focuses on dynamic regret of online convex optimization that c...
read it

Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Nonconvex Optimization
Recently, stochastic momentum methods have been widely adopted in train...
read it

Improved Dropout for Shallow and Deep Learning
Dropout has been witnessed with great success in training deep neural ne...
read it