On the Linear Convergence of Policy Gradient under Hadamard Parameterization

05/31/2023
∙
by   Jiacai Liu, et al.
∙
0
∙

The convergence of deterministic policy gradient under the Hadamard parametrization is studied in the tabular setting and the global linear convergence of the algorithm is established. To this end, we first show that the error decreases at an O(1/k) rate for all the iterations. Based on this result, we further show that the algorithm has a faster local linear convergence rate after k_0 iterations, where k_0 is a constant that only depends on the MDP problem and the step size. Overall, the algorithm displays a linear convergence rate for all the iterations with a loose constant than that for the local linear convergence rate.

READ FULL TEXT

Please sign up or login with your details

Continue with:
Or login with email
Enter Password
Re-enter Password

Forgot password? Click here to reset
Success!
Error Icon An error occurred

Sign in with Google

×

Use your Google Account to sign in to DeepAI

×
Pro

Consider DeepAI Pro

Subscribe to DeepAI Pro
DeepAI Pro
Provides a limited generation allowance each month. When exceeded, you are charged overage rates available at deepai.org/pricing. Also includes an ad-free experience and API access. Renews automatically until canceled. Non-refundable.
Subtotal
Total due today

Payment

Add DeepAI credits
DeepAI credits
One-time purchase. Credits are added to your wallet after payment.
Subtotal
Total due today

Payment