Optimistic Temporal Difference Learning for 2048

11/22/2021
by   Hung Guei, et al.
3

Temporal difference (TD) learning and its variants, such as multistage TD (MS-TD) learning and temporal coherence (TC) learning, have been successfully applied to 2048. These methods rely on the stochasticity of the environment of 2048 for exploration. In this paper, we propose to employ optimistic initialization (OI) to encourage exploration for 2048, and empirically show that the learning quality is significantly improved. This approach optimistically initializes the feature weights to very large values. Since weights tend to be reduced once the states are visited, agents tend to explore those states which are unvisited or visited few times. Our experiments show that both TD and TC learning with OI significantly improve the performance. As a result, the network size required to achieve the same performance is significantly reduced. With additional tunings such as expectimax search, multistage learning, and tile-downgrading technique, our design achieves the state-of-the-art performance, namely an average score of 625 377 and a rate of 72 tiles are reached at a rate of 0.02

READ FULL TEXT

page 4

page 10

research
12/21/2022

On Reinforcement Learning for the Game of 2048

2048 is a single-player stochastic puzzle game. This intriguing and addi...
research
10/26/2020

Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis

Variance reduction techniques have been successfully applied to temporal...
research
10/05/2021

A study of first-passage time minimization via Q-learning in heated gridworlds

Optimization of first-passage times is required in applications ranging ...
research
06/11/2021

Preferential Temporal Difference Learning

Temporal-Difference (TD) learning is a general and very useful tool for ...
research
11/15/2018

Context-Dependent Upper-Confidence Bounds for Directed Exploration

Directed exploration strategies for reinforcement learning are critical ...
research
07/01/2021

Prediction of tone detection thresholds in interaurally delayed noise based on interaural phase difference fluctuations

Differences between the interaural phase of a noise and a target tone im...
research
10/02/2018

Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation

We present a method for Temporal Difference (TD) learning that addresses...

Please sign up or login with your details

Forgot password? Click here to reset