Log In Sign Up

Optimization and passive flow control using single-step deep reinforcement learning

by   H. Ghraieb, et al.

This research gauges the ability of deep reinforcement learning (DRL) techniques to assist the optimization and control of fluid mechanical systems. It combines a novel, "degenerate" version of the proximal policy optimization (PPO) algorithm, that trains a neural network in optimizing the system only once per learning episode, and an in-house stabilized finite elements environment implementing the variational multiscale (VMS) method, that computes the numerical reward fed to the neural network. Three prototypical examples of separated flows in two dimensions are used as testbed for developing the methodology, each of which adds a layer of complexity due either to the unsteadiness of the flow solutions, or the sharpness of the objective function, or the dimension of the control parameter space. Relevance is carefully assessed by comparing systematically to reference data obtained by canonical direct and adjoint methods. Beyond adding value to the shallow literature on this subject, these findings establish the potential of single-step PPO for reliable black-box optimization of computational fluid dynamics (CFD) systems, which paves the way for future progress in optimal flow control using this new class of methods.


page 10

page 19

page 21

page 23


A review on Deep Reinforcement Learning for Fluid Mechanics

Deep reinforcement learning (DRL) has recently been adopted in a wide ra...

Direct shape optimization through deep reinforcement learning

Deep Reinforcement Learning (DRL) has recently spread into a range of do...

Control of a fly-mimicking flyer in complex flow using deep reinforcement learning

An integrated framework of computational fluid-structural dynamics (CFD-...

Deep Q-Network with Proximal Iteration

We employ Proximal Iteration for value-function optimization in reinforc...

The Optimization of the Constant Flow Parallel Micropump Using RBF Neural Network

The objective of this work is to optimize the performance of a constant ...

A statistical learning strategy for closed-loop control of fluid flows

This work discusses a closed-loop control strategy for complex systems u...