Modularization of End-to-End Learning: Case Study in Arcade Games

01/27/2019
by   Andrew Melnik, et al.
0

Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on mastering different types of interactions in a broad range of environments. The modules utilize regression, supervised learning, and reinforcement learning algorithms. Results of this case study in different Atari games suggest that human-level performance can be achieved by a learning agent within a human amount of game experience (10-15 minutes game time) when a proper decomposition of an environment or a task is provided. However, automatization of such decomposition remains a challenging problem. This case study shows how a model of a causal structure underlying an environment or a task can benefit learning time and generalization capability of the agent, and argues in favor of exploiting modular structure in contrast to using pure end-to-end learning approaches.

READ FULL TEXT
research
08/04/2023

A Controllable Co-Creative Agent for Game System Design

Many advancements have been made in procedural content generation for ga...
research
03/12/2020

The Chef's Hat Simulation Environment for Reinforcement-Learning-Based Agents

To achieve social interactions within Human-Robot Interaction (HRI) envi...
research
11/07/2017

Can Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?

Deep reinforcement learning has achieved many recent successes, but our ...
research
02/21/2020

Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning

In many vision-based reinforcement learning (RL) problems, the agent con...
research
09/09/2021

Self-supervised Reinforcement Learning with Independently Controllable Subgoals

To successfully tackle challenging manipulation tasks, autonomous agents...
research
04/19/2022

Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction

In this work we give a case study of an embodied machine-learning (ML) p...
research
02/26/2018

Disentangling the independently controllable factors of variation by interacting with the world

It has been postulated that a good representation is one that disentangl...

Please sign up or login with your details

Forgot password? Click here to reset