A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization

02/20/2023

∙

by   Tesi Xiao, et al.

∙

0

∙

share

We focus on decentralized stochastic non-convex optimization, where n agents work together to optimize a composite objective function which is a sum of a smooth term and a non-smooth convex term. To solve this problem, we propose two single-time scale algorithms: Prox-DASA and Prox-DASA-GT. These algorithms can find ϵ-stationary points in 𝒪(n^-1ϵ^-2) iterations using constant batch sizes (i.e., 𝒪(1)). Unlike prior work, our algorithms achieve a comparable complexity result without requiring large batch sizes, more complex per-iteration operations (such as double loops), or stronger assumptions. Our theoretical findings are supported by extensive numerical experiments, which demonstrate the superiority of our algorithms over previous approaches.