Learning Action Representations for Reinforcement Learning

by   Yash Chandak, et al.
University of Massachusetts Amherst

Most model-free reinforcement learning methods leverage state representations (embeddings) for generalization, but either ignore structure in the space of actions or assume the structure is provided a priori. We show how a policy can be decomposed into a component that acts in a low-dimensional space of action representations and a component that transforms these representations into actual actions. These representations improve generalization over large, finite action sets by allowing the agent to infer the outcomes of actions similar to actions already taken. We provide an algorithm to both learn and use action representations and provide conditions for its convergence. The efficacy of the proposed method is demonstrated on large-scale real-world problems.


page 1

page 2

page 3

page 4


Joint State-Action Embedding for Efficient Reinforcement Learning

While reinforcement learning has achieved considerable successes in rece...

The Natural Language of Actions

We introduce Act2Vec, a general framework for learning context-based act...

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

Deep reinforcement learning (DRL) breaks through the bottlenecks of trad...

DCT: Dual Channel Training of Action Embeddings for Reinforcement Learning with Large Discrete Action Spaces

The ability to learn robust policies while generalizing over large discr...

The Thing That We Tried Didn't Work Very Well : Deictic Representation in Reinforcement Learning

Most reinforcement learning methods operate on propositional representat...

Long-Term Planning and Situational Awareness in OpenAI Five

Understanding how knowledge about the world is represented within model-...

Lifelong Learning with a Changing Action Set

In many real-world sequential decision making problems, the number of av...

Please sign up or login with your details

Forgot password? Click here to reset