Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning

by   Luca Biferale, et al.

To find the path that minimizes the time to navigate between two given points in a fluid flow is known as the Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel which has a slip velocity with fixed intensity, V_s, but variable direction and navigating in a 2D turbulent sea. We use an Actor-Critic RL algorithm, and compare the results with strategies obtained analytically from continuous Optimal Navigation (ON) protocols. We show that for our application, ON solutions are unstable for the typical duration of the navigation process, and are therefore not useful in practice. On the other hand, RL solutions are much more robust with respect to small changes in the initial conditions and to external noise, and are able to find optimal trajectories even when V_s is much smaller than the maximum flow velocity. Furthermore, we show how the RL approach is able to take advantage of the flow properties in order to reach the target, especially when the steering speed is small.


page 2

page 4

page 5

page 6


Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning

We present theoretical and numerical results concerning the problem to f...

Learning Efficient Navigation in Vortical Flow Fields

Efficient point-to-point navigation in the presence of a background flow...

Stabilising viscous extensional flows using Reinforcement Learning

The four-roll mill, wherein four identical cylinders undergo rotation of...

Learning to swim efficiently in a nonuniform flow field

Microswimmers can acquire information on the surrounding fluid by sensin...

Path-planning microswimmers can swim efficiently in turbulent flows

We develop an adversarial-reinforcement learning scheme for microswimmer...

Reinforcement Learning for Active Flow Control in Experiments

We demonstrate experimentally the feasibility of applying reinforcement ...

Local Navigation and Docking of an Autonomous Robot Mower using Reinforcement Learning and Computer Vision

We demonstrate a successful navigation and docking control system for th...

Please sign up or login with your details

Forgot password? Click here to reset