Depth Control of Model-Free AUVs via Reinforcement Learning

11/22/2017
by   Hui Wu, et al.
0

In this paper, we consider depth control problems of an autonomous underwater vehicle (AUV) for tracking the desired depth trajectories. Due to the unknown dynamical model of the AUV, the problems cannot be solved by most of model-based controllers. To this purpose, we formulate the depth control problems of the AUV as continuous-state, continuous-action Markov decision processes (MDPs) under unknown transition probabilities. Based on deterministic policy gradient (DPG) and neural network approximation, we propose a model-free reinforcement learning (RL) algorithm that learns a state-feedback controller from sampled trajectories of the AUV. To improve the performance of the RL algorithm, we further propose a batch-learning scheme through replaying previous prioritized trajectories. We illustrate with simulations that our model-free method is even comparable to the model-based controllers as LQI and NMPC. Moreover, we validate the effectiveness of the proposed RL algorithm on a seafloor data set sampled from the South China Sea.

READ FULL TEXT
research
05/01/2019

Efficient Model-free Reinforcement Learning in Metric Spaces

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Wa...
research
11/27/2019

A selected review on reinforcement learning based control for autonomous underwater vehicles

Recently, reinforcement learning (RL) has been extensively studied and a...
research
01/31/2022

Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control

Reinforcement learning (RL) is a promising, upcoming topic in automatic ...
research
11/16/2021

Analysis of Model-Free Reinforcement Learning Control Schemes on self-balancing Wheeled Extendible System

Traditional linear control strategies have been extensively researched a...
research
04/01/2021

Optimization Algorithm for Feedback and Feedforward Policies towards Robot Control Robust to Sensing Failures

Model-free or learning-based control, in particular, reinforcement learn...
research
04/11/2018

DORA The Explorer: Directed Outreaching Reinforcement Action-Selection

Exploration is a fundamental aspect of Reinforcement Learning, typically...
research
04/17/2018

Regret Bounds for Model-Free Linear Quadratic Control

Model-free approaches for reinforcement learning (RL) and continuous con...

Please sign up or login with your details

Forgot password? Click here to reset