DeepAI AI Chat
Log In Sign Up

Reinforcement learning based adaptive metaheuristics

by   Michele Tessari, et al.
Università di Trento

Parameter adaptation, that is the capability to automatically adjust an algorithm's hyperparameters depending on the problem being faced, is one of the main trends in evolutionary computation applied to numerical optimization. While several handcrafted adaptation policies have been proposed over the years to address this problem, only few attempts have been done so far at applying machine learning to learn such policies. Here, we introduce a general-purpose framework for performing parameter adaptation in continuous-domain metaheuristics based on state-of-the-art reinforcement learning algorithms. We demonstrate the applicability of this framework on two algorithms, namely Covariance Matrix Adaptation Evolution Strategies (CMA-ES) and Differential Evolution (DE), for which we learn, respectively, adaptation policies for the step-size (for CMA-ES), and the scale factor and crossover rate (for DE). We train these policies on a set of 46 benchmark functions at different dimensionalities, with various inputs to the policies, in two settings: one policy per function, and one global policy for all functions. Compared, respectively, to the Cumulative Step-size Adaptation (CSA) policy and to two well-known adaptive DE variants (iDE and jDE), our policies are able to produce competitive results in the majority of cases, especially in the case of DE.


Cumulative Step-size Adaptation on Linear Functions

The CSA-ES is an Evolution Strategy with Cumulative Step size Adaptation...

On-Policy Trust Region Policy Optimisation with Replay Buffers

Building upon the recent success of deep reinforcement learning methods,...

Markov Chain Analysis of Evolution Strategies on a Linear Constraint Optimization Problem

This paper analyses a (1,λ)-Evolution Strategy, a randomised comparison-...

TPAM: A Simulation-Based Model for Quantitatively Analyzing Parameter Adaptation Methods

While a large number of adaptive Differential Evolution (DE) algorithms ...

Analyzing Adaptive Parameter Landscapes in Parameter Adaptation Methods for Differential Evolution

Since the scale factor and the crossover rate significantly influence th...

Sparsity Inducing Representations for Policy Decompositions

Policy Decomposition (PoDec) is a framework that lessens the curse of di...