Minimizing Nonsmooth Convex Functions with Variable Accuracy

by   Natasa Krejic, et al.

We consider unconstrained optimization problems with nonsmooth and convex objective function in the form of mathematical expectation. The proposed method approximates the objective function with a sample average function by using different sample size in each iteration. The sample size is chosen in an adaptive manner based on the Inexact Restoration. The method uses line search and assumes descent directions with respect to the current approximate function. We prove the almost sure convergence under the standard assumptions. The convergence rate is also considered and the worst-case complexity of π’ͺ (Ξ΅^-2) is proved. Numerical results for two types of problems, machine learning hinge loss and stochastic linear complementarity problems, show the efficiency of the proposed scheme.



There are no comments yet.


page 1

page 2

page 3

page 4

βˆ™ 03/23/2022

Spectral Projected Subgradient Method for Nonsmooth Convex Optimization Problems

We consider constrained optimization problems with a nonsmooth objective...
βˆ™ 07/07/2021

A stochastic first-order trust-region method with inexact restoration for finite-sum minimization

We propose a stochastic first-order trust-region method with inexact fun...
βˆ™ 07/31/2020

LSOS: Line-search Second-Order Stochastic optimization methods

We develop a line-search second-order algorithmic framework for optimiza...
βˆ™ 05/25/2020

Reactive Sample Size for Heuristic Search in Simulation-based Optimization

In simulation-based optimization, the optimal setting of the input param...
βˆ™ 10/30/2017

Adaptive Sampling Strategies for Stochastic Optimization

In this paper, we propose a stochastic optimization method that adaptive...
βˆ™ 11/09/2018

Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization

A regularization algorithm using inexact function values and inexact der...
βˆ™ 12/07/2020

Adaptive Sequential SAA for Solving Two-stage Stochastic Linear Programs

We present adaptive sequential SAA (sample average approximation) algori...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.