Expanded Alternating Optimization of Nonconvex Functions with Applications to Matrix Factorization and Penalized Regression

12/12/2014
by   W. James Murdoch, et al.
0

We propose a general technique for improving alternating optimization (AO) of nonconvex functions. Starting from the solution given by AO, we conduct another sequence of searches over subspaces that are both meaningful to the optimization problem at hand and different from those used by AO. To demonstrate the utility of our approach, we apply it to the matrix factorization (MF) algorithm for recommender systems and the coordinate descent algorithm for penalized regression (PR), and show meaningful improvements using both real-world (for MF) and simulated (for PR) data sets. Moreover, we demonstrate for MF that, by constructing search spaces customized to the given data set, we can significantly increase the convergence rate of our technique.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/17/2018

Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation

In this paper, we propose a general framework to accelerate significantl...
research
05/11/2023

Convergence of Alternating Gradient Descent for Matrix Factorization

We consider alternating gradient descent (AGD) with fixed step size η > ...
research
03/24/2017

A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality

Symmetric nonnegative matrix factorization (SymNMF) has important applic...
research
05/18/2022

Efficient Mixed Dimension Embeddings for Matrix Factorization

Despite the prominence of neural network approaches in the field of reco...
research
09/18/2021

Coordinate Descent for MCP/SCAD Penalized Least Squares Converges Linearly

Recovering sparse signals from observed data is an important topic in si...
research
10/19/2017

Efficient Robust Matrix Factorization with Nonconvex Penalties

Robust matrix factorization (RMF) is a fundamental tool with lots of app...
research
10/19/2017

Scalable Robust Matrix Factorization with Nonconvex Loss

Robust matrix factorization (RMF), which uses the ℓ_1-loss, often outper...

Please sign up or login with your details

Forgot password? Click here to reset