AI Chat AI Image Generator AI Video Text to Speech

Online Dual Coordinate Ascent Learning

02/24/2016

∙

by Bicheng Ying, et al.

∙

∙

The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finite sample sizes and relies on performing multiple passes over the same data. This formulation is not well-suited for online implementations where data keep streaming in. In this work, we develop an online dual coordinate-ascent (O-DCA) algorithm that is able to respond to streaming data and does not need to revisit the past data. This feature embeds the resulting construction with continuous adaptation, learning, and tracking abilities, which are particularly attractive for online learning scenarios.

Bicheng Ying
15 publications
Kun Yuan
47 publications
Ali H. Sayed
89 publications

page 1

page 2

page 3

page 4

research

∙ 09/10/2012

Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization

Stochastic Gradient Descent (SGD) has become popular for solving large s...

0 Shai Shalev-Shwartz, et al. ∙

research

∙ 03/02/2016

Without-Replacement Sampling for Stochastic Gradient Methods: Convergence Results and Application to Distributed Optimization

Stochastic gradient methods for machine learning and optimization proble...

0 Ohad Shamir, et al. ∙

research

∙ 02/21/2020

Debiasing Stochastic Gradient Descent to handle missing values

A major caveat of large scale data is their incom-pleteness. We propose ...

0 Aude Sportisse, et al. ∙

research

∙ 11/07/2017

FADO: A Deterministic Detection/Learning Algorithm

This paper proposes and studies a detection technique for adversarial sc...

0 Kristiaan Pelckmans, et al. ∙

research

∙ 09/22/2015

Stochastic gradient descent methods for estimation with large data sets

We develop methods for parameter estimation in settings with large-scale...

0 Dustin Tran, et al. ∙

research

∙ 02/05/2023

Scalable inference in functional linear regression with streaming data

Traditional static functional data analysis is facing new challenges due...

0 Jinhan Xie, et al. ∙

research

∙ 04/13/2016

A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization

In modern large-scale machine learning applications, the training data a...

0 Shun Zheng, et al. ∙