Multi-Objective Deep Q-Learning with Subsumption Architecture

04/21/2017
by   Tomasz Tajmajer, et al.
0

In this work we present a method for using Deep Q-Networks (DQNs) in multi-objective tasks. Deep Q-Networks provide remarkable performance in single objective tasks learning from high-level visual perception. However, in many scenarios (e.g in robotics), the agent needs to pursue multiple objectives simultaneously. We propose an architecture in which separate DQNs are used to control the agent's behaviour with respect to particular objectives. In this architecture we use signal suppression, known from the (Brooks) subsumption architecture, to combine outputs of several DQNs into a single action. Our architecture enables the decomposition of the agent's behaviour into controllable and replaceable sub-behaviours learned by distinct modules. To evaluate our solution we used a game-like simulator in which an agent - provided with high-level visual input - pursues multiple objectives in a 2D world. Our solution provides benefits of modularity, while its performance is comparable to the monolithic approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/22/2020

pymoo: Multi-objective Optimization in Python

Python has become the programming language of choice for research and in...
research
11/09/2022

Deep W-Networks: Solving Multi-Objective Optimisation Problems With Deep Reinforcement Learning

In this paper, we build on advances introduced by the Deep Q-Networks (D...
research
05/17/2017

Identification and Off-Policy Learning of Multiple Objectives Using Adaptive Clustering

In this work, we present a methodology that enables an agent to make eff...
research
05/15/2020

A Distributional View on Multi-Objective Policy Optimization

Many real-world problems require trading off multiple competing objectiv...
research
06/16/2023

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Existing traffic signal control systems rely on oversimplified rule-base...
research
02/22/2022

Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach

Penetration Testing plays a critical role in evaluating the security of ...
research
04/13/2022

Modularity benefits reinforcement learning agents with competing homeostatic drives

The problem of balancing conflicting needs is fundamental to intelligenc...

Please sign up or login with your details

Forgot password? Click here to reset