Huishuai Zhang

research

∙ 05/23/2023

Selective Pre-training for Private Fine-tuning

Suppose we want to train text prediction models in email clients or word...

0 Da Yu, et al. ∙

research

∙ 11/29/2022

Similarity Distribution based Membership Inference Attack on Person Re-identification

While person Re-identification (Re-ID) has progressed rapidly due to its...

0 Junyao Gao, et al. ∙

research

∙ 06/27/2022

Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization

By ensuring differential privacy in the learning algorithms, one can rig...

0 Xiaodong Yang, et al. ∙

research

∙ 06/09/2022

Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks

Adversarial examples, which are usually generated for specific inputs wi...

0 Huishuai Zhang, et al. ∙

research

∙ 06/06/2022

Per-Instance Privacy Accounting for Differentially Private Stochastic Gradient Descent

Differentially private stochastic gradient descent (DP-SGD) is the workh...

12 Da Yu, et al. ∙

research

∙ 05/22/2022

Robust Quantity-Aware Aggregation for Federated Learning

Federated learning (FL) enables multiple clients to collaboratively trai...

0 Jingwei Yi, et al. ∙

research

∙ 11/01/2021

Indiscriminate Poisoning Attacks Are Shortcuts

Indiscriminate data poisoning attacks, which add imperceptible perturbat...

0 Da Yu, et al. ∙

research

∙ 10/13/2021

Differentially Private Fine-tuning of Language Models

We give simpler, sparser, and faster algorithms for differentially priva...

0 Da Yu, et al. ∙

research

∙ 10/08/2021

Momentum Doesn't Change the Implicit Bias

The momentum acceleration technique is widely adopted in many optimizati...

0 Bohan Wang, et al. ∙

research

∙ 06/29/2021

Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit

Balancing exploration and exploitation (EE) is a fundamental problem in ...

0 Yichi Zhou, et al. ∙

research

∙ 06/17/2021

Large Scale Private Learning via Low-rank Reparametrization

We propose a reparametrization scheme to address the challenges of apply...

0 Da Yu, et al. ∙

research

∙ 05/31/2021

Adversarial Training with Rectified Rejection

Adversarial training (AT) is one of the most effective strategies for pr...

0 Tianyu Pang, et al. ∙

research

∙ 02/25/2021

Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning

The privacy leakage of the model about the training data can be bounded ...

0 Da Yu, et al. ∙

research

∙ 01/08/2021

BN-invariant sharpness regularizes the training model to better generalization

It is arguably believed that flatter minima can generalize better. Howev...

0 Mingyang Yi, et al. ∙

research

∙ 08/04/2020

Well-Conditioned Methods for Ill-Conditioned Systems: Linear Regression with Semi-Random Noise

Classical iterative algorithms for linear system solving and regression ...

0 Jerry Li, et al. ∙

research

∙ 07/21/2020

Membership Inference with Privately Augmented Data Endorses the Benign while Suppresses the Adversary

Membership inference (MI) in machine learning decides whether a given ex...

11 Da Yu, et al. ∙

research

∙ 06/29/2020

Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia

Adaptive Momentum Estimation (Adam), which combines Adaptive Learning Ra...

8 Zeke Xie, et al. ∙

research

∙ 02/12/2020

On Layer Normalization in the Transformer Architecture

The Transformer is widely used in natural language processing tasks. To ...

0 Ruibin Xiong, et al. ∙

research

∙ 11/26/2019

Gradient Perturbation is Underrated for Differentially Private Convex Optimization

Gradient perturbation, widely used for differentially private optimizati...

0 Da Yu, et al. ∙

research

∙ 05/29/2019

Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data

Stochastic variance reduced methods have gained a lot of interest recent...

0 Shicong Cen, et al. ∙

research

∙ 03/17/2019

Training Over-parameterized Deep ResNet Is almost as Easy as Training a Two-layer Network

It has been proved that gradient descent converges linearly to the globa...

0 Huishuai Zhang, et al. ∙

research

∙ 01/02/2019

SGD Converges to Global Minimum in Deep Learning via Star-convex Path

Stochastic gradient descent (SGD) has been found to be surprisingly effe...

18 Yi Zhou, et al. ∙

research

∙ 09/19/2018

Capacity Control of ReLU Neural Networks by Basis-path Norm

Recently, path norm was proposed as a new capacity measure for neural ne...

0 Shuxin Zheng, et al. ∙

research

∙ 02/27/2018

Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation

Stochastic gradient descent (SGD) has achieved great success in training...

0 Huishuai Zhang, et al. ∙

research

∙ 02/19/2018

Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization

The success of deep learning has led to a rising interest in the general...

0 Yi Zhou, et al. ∙

research

∙ 12/20/2017

Block-diagonal Hessian-free Optimization for Training Neural Networks

Second-order methods for neural network optimization have several advant...

0 Huishuai Zhang, et al. ∙

research

∙ 09/23/2017

Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent

Recent work has demonstrated the effectiveness of gradient descent for d...

0 Yuanxin Li, et al. ∙

research

∙ 05/25/2016

Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations

We study the phase retrieval problem, which solves quadratic system of e...

0 Huishuai Zhang, et al. ∙

research

∙ 03/11/2016

Median-Truncated Nonconvex Approach for Phase Retrieval with Outliers

This paper investigates the phase retrieval problem, which aims to recov...

0 Huishuai Zhang, et al. ∙

Huishuai Zhang

Featured Co-authors