DeepAI AI Chat
Log In Sign Up

Optimistic Temporal Difference Learning for 2048

by   Hung Guei, et al.

Temporal difference (TD) learning and its variants, such as multistage TD (MS-TD) learning and temporal coherence (TC) learning, have been successfully applied to 2048. These methods rely on the stochasticity of the environment of 2048 for exploration. In this paper, we propose to employ optimistic initialization (OI) to encourage exploration for 2048, and empirically show that the learning quality is significantly improved. This approach optimistically initializes the feature weights to very large values. Since weights tend to be reduced once the states are visited, agents tend to explore those states which are unvisited or visited few times. Our experiments show that both TD and TC learning with OI significantly improve the performance. As a result, the network size required to achieve the same performance is significantly reduced. With additional tunings such as expectimax search, multistage learning, and tile-downgrading technique, our design achieves the state-of-the-art performance, namely an average score of 625 377 and a rate of 72 tiles are reached at a rate of 0.02


page 4

page 10


On Reinforcement Learning for the Game of 2048

2048 is a single-player stochastic puzzle game. This intriguing and addi...

Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis

Variance reduction techniques have been successfully applied to temporal...

A study of first-passage time minimization via Q-learning in heated gridworlds

Optimization of first-passage times is required in applications ranging ...

Preferential Temporal Difference Learning

Temporal-Difference (TD) learning is a general and very useful tool for ...

Context-Dependent Upper-Confidence Bounds for Directed Exploration

Directed exploration strategies for reinforcement learning are critical ...

Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation

We present a method for Temporal Difference (TD) learning that addresses...