Solving the Order Batching and Sequencing Problem using Deep Reinforcement Learning

by   Bram Cals, et al.

In e-commerce markets, on time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach for deciding how and when orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so with which other orders. We approach the problem by formulating it as a semi-Markov decision process and develop a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach is able to develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics.


page 8

page 9


Interpretable Deep Reinforcement Learning for Green Security Games with Real-Time Information

Green Security Games with real-time information (GSG-I) add the real-tim...

A Deep Reinforcement Learning Approach for Constrained Online Logistics Route Assignment

As online shopping prevails and e-commerce platforms emerge, there is a ...

A State Aggregation Approach for Solving Knapsack Problem with Deep Reinforcement Learning

This paper proposes a Deep Reinforcement Learning (DRL) approach for sol...

Solving the Traveling Salesperson Problem with Precedence Constraints by Deep Reinforcement Learning

This work presents solutions to the Traveling Salesperson Problem with p...

Automating DBSCAN via Deep Reinforcement Learning

DBSCAN is widely used in many scientific and engineering fields because ...

Visualizing Dynamics: from t-SNE to SEMI-MDPs

Deep Reinforcement Learning (DRL) is a trending field of research, showi...

Please sign up or login with your details

Forgot password? Click here to reset