The Point to Which Soft Actor-Critic Converges

03/01/2023
by   Jianfei Ma, et al.
0

Soft actor-critic is a successful successor over soft Q-learning. While lived under maximum entropy framework, their relationship is still unclear. In this paper, we prove that in the limit they converge to the same solution. This is appealing since it translates the optimization from an arduous to an easier way. The same justification can also be applied to other regularizers such as KL divergence.

READ FULL TEXT

page 1

page 2

page 3

research
12/31/2021

Actor Loss of Soft Actor Critic Explained

This technical report is devoted to explaining how the actor loss of sof...
research
09/27/2022

Regularized Soft Actor-Critic for Behavior Transfer Learning

Existing imitation learning methods mainly focus on making an agent effe...
research
02/07/2022

Soft Actor-Critic with Inhibitory Networks for Faster Retraining

Reusing previously trained models is critical in deep reinforcement lear...
research
06/17/2019

Hierarchical Soft Actor-Critic: Adversarial Exploration via Mutual Information Optimization

We describe a novel extension of soft actor-critics for hierarchical Dee...
research
12/08/2021

Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization

Hyper-parameter optimization is a crucial problem in machine learning as...
research
01/09/2022

An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic

Recently, the applications of deep neural network (DNN) have been very p...
research
05/19/2023

Regularization of Soft Actor-Critic Algorithms with Automatic Temperature Adjustment

This work presents a comprehensive analysis to regularize the Soft Actor...

Please sign up or login with your details

Forgot password? Click here to reset