MLKX
Machine Learning Knowledge Exchange
view repo
Deep reinforcement learning (deep RL) has been successful in learning sophisticated behaviors automatically; however, the learning process requires a huge number of trials. In contrast, animals can learn new tasks in just a few trials, benefiting from their prior knowledge about the world. This paper seeks to bridge this gap. Rather than designing a "fast" reinforcement learning algorithm, we propose to represent it as a recurrent neural network (RNN) and learn it from data. In our proposed method, RL^2, the algorithm is encoded in the weights of the RNN, which are learned slowly through a general-purpose ("slow") RL algorithm. The RNN receives all information a typical RL algorithm would receive, including observations, actions, rewards, and termination flags; and it retains its state across episodes in a given Markov Decision Process (MDP). The activations of the RNN store the state of the "fast" RL algorithm on the current (previously unseen) MDP. We evaluate RL^2 experimentally on both small-scale and large-scale problems. On the small-scale side, we train it to solve randomly generated multi-arm bandit problems and finite MDPs. After RL^2 is trained, its performance on new MDPs is close to human-designed algorithms with optimality guarantees. On the large-scale side, we test RL^2 on a vision-based navigation task and show that it scales up to high-dimensional problems.
READ FULL TEXT
The standard RL world model is that of a Markov Decision Process (MDP). ...
read it
Exogenous state variables and rewards can slow down reinforcement learni...
read it
This paper addresses the general problem of reinforcement learning (RL) ...
read it
The use of Reinforcement Learning in real-world scenarios is strongly li...
read it
Reinforcement learning agents usually learn from scratch, which requires...
read it
This work applies reinforcement learning (RL) from the AI machine learni...
read it
We consider the Markov Decision Process (MDP) of selecting a subset of i...
read it
Machine Learning Knowledge Exchange
Comments
There are no comments yet.