Smart Train Operation Algorithms based on Expert Knowledge and Reinforcement Learning

03/06/2020
by   Rui Zhou, et al.
0

During recent decades, the automatic train operation (ATO) system has been gradually adopted in many subway systems. On the one hand, it is more intelligent than traditional manual driving; on the other hand, it increases the energy consumption and decreases the riding comfort of the subway system. This paper proposes two smart train operation algorithms based on the combination of expert knowledge and reinforcement learning algorithms. Compared with previous works, smart train operation algorithms can realize the control of continuous action for the subway system and satisfy multiple objectives (the safety, the punctuality, the energy efficiency, and the riding comfort) without using an offline optimized speed profile. Firstly, through analyzing historical data of experienced subway drivers, we summarize the expert knowledge rules and build inference methods to guarantee the riding comfort, the punctuality and the safety of the subway system. Then we develop two algorithms to realize the control of continuous action and to ensure the energy efficiency of train operation. Among them, one is the smart train operation (STO) algorithm based on deep deterministic policy gradient named (STOD) and another is the smart train operation algorithm based on normalized advantage function (STON). Finally, we verify the performance of proposed algorithms via some numerical simulations with the real field data collected from the Yizhuang Line of the Beijing Subway and their performance will be compared with existing ATO algorithms. The results of numerical simulations show that the developed smart train operation systems are better than manual driving and existing ATO algorithms in respect of energy efficiency. In addition, STOD and STON have the ability to adapt to different trip times and different resistance conditions.

READ FULL TEXT
research
07/16/2020

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Development of hybrid electric vehicles depends on an advanced and effic...
research
04/30/2023

ReLBOT: A Transfer Learning Approach to Minimize Reinforcement Learning Risks in Smart Buildings

Smart buildings aim to optimize energy consumption by applying artificia...
research
08/22/2020

MLD: An Intelligent Memory Leak Detection Scheme Based on Defect Modes in Smart Grids

With the expansion of the software scale and complexity of smart grid sy...
research
08/08/2018

A Hybrid Dynamic-regenerative Damping Scheme for Energy Regeneration in Variable Impedance Actuators

Increasing research efforts have been made to improve the energy efficie...
research
08/18/2023

Integrating Expert Guidance for Efficient Learning of Safe Overtaking in Autonomous Driving Using Deep Reinforcement Learning

Overtaking on two-lane roads is a great challenge for autonomous vehicle...
research
03/09/2018

Valuing knowledge, information and agency in Multi-agent Reinforcement Learning: a case study in smart buildings

Increasing energy efficiency in buildings can reduce costs and emissions...

Please sign up or login with your details

Forgot password? Click here to reset