Parallel sequential Monte Carlo for stochastic optimization

11/23/2018
by   Ömer Deniz Akyıldız, et al.
0

We propose a parallel sequential Monte Carlo optimization method to minimize cost functions which are computed as the sum of many component functions. The proposed scheme is a stochastic zeroth order optimization algorithm which uses only evaluations of small subsets of component functions to collect information from the problem. The algorithm consists of a bank of samplers and generates particle approximations of several sequences of probability measures. These measures are constructed in such a way that they have associated probability density functions whose global maxima coincide with the global minima of the cost function. The algorithm selects the best performing sampler and uses it to approximate a global minimum of the cost function. We prove analytically that the resulting estimator converges to a global minimum of the cost function almost surely as the number of Monte Carlo samples tends to infinity. We show that the algorithm can tackle cost functions with multiple minima or with wide flat regions.

READ FULL TEXT
research
05/24/2018

Strategic Monte Carlo Methods for State and Parameter Estimation in High Dimensional Nonlinear Problems

In statistical data assimilation one seeks the largest maximum of the co...
research
05/25/2023

How to escape sharp minima

Modern machine learning applications have seen a remarkable success of o...
research
05/24/2019

Computational cost for determining an approximate global minimum using the selection and crossover algorithm

This work examines the expected computational cost to determine an appro...
research
07/18/2017

Coresets for Triangulation

Multiple-view triangulation by ℓ_∞ minimisation has become established i...
research
09/14/2023

Landscape-Sketch-Step: An AI/ML-Based Metaheuristic for Surrogate Optimization Problems

In this paper, we introduce a new heuristics for global optimization in ...
research
08/21/2018

Search for Common Minima in Joint Optimization of Multiple Cost Functions

We present a novel optimization method, named the Combined Optimization ...
research
08/02/2023

A Fast Monte Carlo algorithm for evaluating matrix functions with application in complex networks

We propose a novel stochastic algorithm that randomly samples entire row...

Please sign up or login with your details

Forgot password? Click here to reset