Nonconvex Stochastic Nested Optimization via Stochastic ADMM

11/12/2019
by   Zhongruo Wang, et al.
0

We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We proposed the stochastic ADMM to solve this complicated objective. In order to find an ϵ stationary point where the expected norm of the subgradient of corresponding augmented Lagrangian is smaller than ϵ, the total sample complexity of our method is O(ϵ^-3) for the online case and O((2N_1 + N_2) + (2N_1 + N_2)^1/2ϵ^-2) for the finite sum case. The computational complexity is consistent with proximal version proposed in <cit.>, but our algorithm can solve more general problem when the proximal mapping of the penalty is not easy to compute.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/29/2019

Multi-Level Composite Stochastic Optimization via Nested Variance Reduction

We consider multi-level composite optimization problems where each mappi...
research
07/25/2016

Accelerating Stochastic Composition Optimization

Consider the stochastic composition optimization problem where the objec...
research
08/24/2020

Stochastic Multi-level Composition Optimization Algorithms with Level-Independent Convergence Rates

In this paper, we study smooth stochastic multi-level composition optimi...
research
05/11/2017

Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization

We consider the stochastic composition optimization problem proposed in ...
research
12/16/2013

Adaptive Stochastic Alternating Direction Method of Multipliers

The Alternating Direction Method of Multipliers (ADMM) has been studied ...
research
12/19/2022

Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization

Many real-world problems not only have complicated nonconvex functional ...
research
06/22/2021

A stochastic linearized proximal method of multipliers for convex stochastic optimization with expectation constraints

This paper considers the problem of minimizing a convex expectation func...

Please sign up or login with your details

Forgot password? Click here to reset