DeepAI AI Chat
Log In Sign Up

Strategic bidding in freight transport using deep reinforcement learning

by   Wouter van Heeswijk, et al.

This paper presents a multi-agent reinforcement learning algorithm to represent strategic bidding behavior in freight transport markets. Using this algorithm, we investigate whether feasible market equilibriums arise without any central control or communication between agents. Studying behavior in such environments may serve as a stepping stone towards self-organizing logistics systems like the Physical Internet. We model an agent-based environment in which a shipper and a carrier actively learn bidding strategies using policy gradient methods, posing bid- and ask prices at the individual container level. Both agents aim to learn the best response given the expected behavior of the opposing agent. A neutral broker allocates jobs based on bid-ask spreads. Our game-theoretical analysis and numerical experiments focus on behavioral insights. To evaluate system performance, we measure adherence to Nash equilibria, fairness of reward division and utilization of transport capacity. We observe good performance both in predictable, deterministic settings ( 95 adherence to Nash equilibria) and highly stochastic environments ( 85 adherence). Risk-seeking behavior may increase an agent's reward share, as long as the strategies are not overly aggressive. The results suggest a potential for full automation and decentralization of freight transport markets.


Computational Performance of Deep Reinforcement Learning to find Nash Equilibria

We test the performance of deep deterministic policy gradient (DDPG), a ...

Smart Containers With Bidding Capacity: A Policy Gradient Algorithm for Semi-Cooperative Learning

Smart modular freight containers – as propagated in the Physical Interne...

Deep reinforcement learning of event-triggered communication and control for multi-agent cooperative transport

In this paper, we explore a multi-agent reinforcement learning approach ...

Resolving Implicit Coordination in Multi-Agent Deep Reinforcement Learning with Deep Q-Networks Game Theory

We address two major challenges of implicit coordination in multi-agent ...

Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisions

We formulate computation offloading as a decentralized decision-making p...

Matching Queues, Flexibility and Incentives

Motivated in part by online marketplaces such as ridesharing and freelan...

Extending Environments To Measure Self-Reflection In Reinforcement Learning

We consider an extended notion of reinforcement learning in which the en...