Data-driven Algorithm Selection and Parameter Tuning: Two Case studies in Optimization and Signal Processing

05/31/2019
by   Jesús A. De Loera, et al.
2

Machine learning algorithms typically rely on optimization subroutines and are well-known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimization and signal processing algorithms. As a proof of concept, we use our approach to improve two popular data processing subroutines in data science: stochastic gradient descent and greedy methods in compressed sensing. We provide experimental results that demonstrate the answer is "yes", machine learning algorithms do lead to more effective outcomes for optimization problems, and show the future potential for this research direction.

READ FULL TEXT

page 6

page 7

page 10

research
02/22/2023

Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning

Stochastic approximation (SA) is a classical algorithm that has had sinc...
research
09/30/2016

A Primer on Coordinate Descent Algorithms

This monograph presents a class of algorithms called coordinate descent ...
research
09/12/2020

A general framework for decentralized optimization with first-order methods

Decentralized optimization to minimize a finite sum of functions over a ...
research
04/21/2019

Stability Analysis for a Class of Sparse Optimization Problems

The sparse optimization problems arise in many areas of science and engi...
research
12/28/2018

Signal Classification under structure sparsity constraints

Object Classification is a key direction of research in signal and image...
research
12/22/2021

On Asymptotic Linear Convergence of Projected Gradient Descent for Constrained Least Squares

Many recent problems in signal processing and machine learning such as c...
research
12/18/2014

Stochastic Descent Analysis of Representation Learning Algorithms

Although stochastic approximation learning methods have been widely used...

Please sign up or login with your details

Forgot password? Click here to reset