Entropy Controlled Non-Stationarity for Improving Performance of Independent Learners in Anonymous MARL Settings

03/27/2018
by   Tanvi Verma, et al.
0

With the advent of sequential matching (of supply and demand) systems (uber, Lyft, Grab for taxis; ubereats, deliveroo, etc for food; amazon prime, lazada etc. for groceries) across many online and offline services, individuals (taxi drivers, delivery boys, delivery van drivers, etc.) earn more by being at the "right" place at the "right" time. We focus on learning techniques for providing guidance (on right locations to be at right times) to individuals in the presence of other "learning" individuals. Interactions between indivduals are anonymous, i.e, the outcome of an interaction (competing for demand) is independent of the identity of the agents and therefore we refer to these as Anonymous MARL settings. Existing research of relevance is on independent learning using Reinforcement Learning (RL) or on Multi-Agent Reinforcement Learning (MARL). The number of individuals in aggregation systems is extremely large and individuals have their own selfish interest (of maximising revenue). Therefore, traditional MARL approaches are either not scalable or assumptions of common objective or action coordination are not viable. In this paper, we focus on improving performance of independent reinforcement learners, specifically the popular Deep Q-Networks (DQN) and Advantage Actor Critic (A2C) approaches by exploiting anonymity. Specifically, we control non-stationarity introduced by other agents using entropy of agent density distribution. We demonstrate a significant improvement in revenue for individuals and for all agents together with our learners on a generic experimental set up for aggregation systems and a real world taxi dataset.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/16/2020

Value Variance Minimization for Learning Approximate Equilibrium in Aggregation Systems

For effective matching of resources (e.g., taxis, food, bikes, shopping ...
research
06/11/2020

Distributed Reinforcement Learning in Multi-Agent Networked Systems

We study distributed reinforcement learning (RL) for a network of agents...
research
12/29/2019

Individual specialization in multi-task environments with multiagent reinforcement learners

There is a growing interest in Multi-Agent Reinforcement Learning (MARL)...
research
05/02/2021

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

The bus system is a critical component of sustainable urban transportati...
research
08/11/2019

A Review of Cooperative Multi-Agent Deep Reinforcement Learning

Deep Reinforcement Learning has made significant progress in multi-agent...
research
08/07/2023

AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning

StarCraft II is one of the most challenging simulated reinforcement lear...
research
06/19/2020

Learn to Earn: Enabling Coordination within a Ride Hailing Fleet

The problem of optimizing social welfare objectives on multi sided ride ...

Please sign up or login with your details

Forgot password? Click here to reset