Inertial Stochastic PALM and its Application for Learning Student-t Mixture Models

by   Johannes Hertrich, et al.

Inertial algorithms for minimizing nonsmooth and nonconvex functions as the inertial proximal alternating linearized minimization algorithm (iPALM) have demonstrated their superiority with respect to computation time over their non inertial variants. In many problems in imaging and machine learning, the objective functions have a special form involving huge data which encourage the application of stochastic algorithms. While the stochastic gradient descent algorithm is still used in the majority of applications, recently also stochastic algorithms for minimizing nonsmooth and nonconvex functions were proposed. In this paper, we derive an inertial variant of the SPRING algorithm, called iSPRING, and prove linear convergence of the algorithm under certain assumptions. Numerical experiments show that our new algorithm performs better than SPRING or its deterministic counterparts, although the improvement for the inertial stochastic approach is not as large as those for the inertial deterministic one. The second aim of the paper is to demonstrate that (inertial) PALM both in the deterministic and stochastic form can be used for learning the parameters of Student-t mixture models. We prove that the objective function of such models fulfills all convergence assumptions of the algorithms and demonstrate their performance by numerical examples.


page 1

page 2

page 3

page 4


On the complexity of convex inertial proximal algorithms

The inertial proximal gradient algorithm is efficient for the composite ...

A block inertial Bregman proximal algorithm for nonsmooth nonconvex problems

In this paper, a block inertial Bregman proximal algorithm, namely [], f...

An abstract convergence framework with application to inertial inexact forward–backward methods

In this paper we introduce a novel abstract descent scheme suited for th...

Modeling reverse thinking for machine learning

Human inertial thinking schemes can be formed through learning, which ar...

Inertial Proximal Deep Learning Alternating Minimization for Efficient Neutral Network Training

In recent years, the Deep Learning Alternating Minimization (DLAM), whic...

An Inertial Block Majorization Minimization Framework for Nonsmooth Nonconvex Optimization

In this paper, we introduce TITAN, a novel inerTial block majorIzation m...

Bregman Proximal Framework for Deep Linear Neural Networks

A typical assumption for the analysis of first order optimization method...

Please sign up or login with your details

Forgot password? Click here to reset