When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee

03/01/2022
by   Dixian Zhu, et al.
0

In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning. We propose new formulations of pAUC surrogate objectives by using the distributionally robust optimization (DRO) to define the loss for each individual positive data. We consider two formulations of DRO, one of which is based on conditional-value-at-risk (CVaR) that yields a non-smooth but exact estimator for pAUC, and another one is based on a KL divergence regularized DRO that yields an inexact but smooth (soft) estimator for pAUC. For both one-way and two-way pAUC maximization, we propose two algorithms and prove their convergence for optimizing their two formulations, respectively. Experiments demonstrate the effectiveness of the proposed algorithms for pAUC maximization for deep learning on various datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/28/2019

Stochastic AUC Maximization with Deep Neural Networks

Stochastic AUC maximization has garnered an increasing interest due to b...
research
03/03/2022

Large-scale Optimization of Partial AUC in a Range of False Positive Rates

The area under the ROC curve (AUC) is one of the most widely used perfor...
research
06/12/2020

Fast Objective and Duality Gap Convergence for Non-convex Strongly-concave Min-max Problems

This paper focuses on stochastic methods for solving smooth non-convex s...
research
05/05/2020

Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks

In this paper, we study distributed algorithms for large-scale AUC maxim...
research
07/08/2022

Balanced Self-Paced Learning for AUC Maximization

Learning to improve AUC performance is an important topic in machine lea...
research
10/08/2022

Asymptotically Unbiased Instance-wise Regularized Partial AUC Optimization: Theory and Algorithm

The Partial Area Under the ROC Curve (PAUC), typically including One-way...
research
02/21/2020

Efficient model-based Bioequivalence Testing

The classical approach to analyze pharmacokinetic (PK) data in bioequiva...

Please sign up or login with your details

Forgot password? Click here to reset