Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environment

by   Jivitesh Sharma, et al.

We focus on the important problem of emergency evacuation, which clearly could benefit from reinforcement learning that has been largely unaddressed. Emergency evacuation is a complex task which is difficult to solve with reinforcement learning, since an emergency situation is highly dynamic, with a lot of changing variables and complex constraints that makes it difficult to train on. In this paper, we propose the first fire evacuation environment to train reinforcement learning agents for evacuation planning. The environment is modelled as a graph capturing the building structure. It consists of realistic features like fire spread, uncertainty and bottlenecks. We have implemented the environment in the OpenAI gym format, to facilitate future research. We also propose a new reinforcement learning approach that entails pretraining the network weights of a DQN based agents to incorporate information on the shortest path to the exit. We achieved this by using tabular Q-learning to learn the shortest path on the building model's graph. This information is transferred to the network by deliberately overfitting it on the Q-matrix. Then, the pretrained DQN model is trained on the fire evacuation environment to generate the optimal evacuation path under time varying conditions. We perform comparisons of the proposed approach with state-of-the-art reinforcement learning algorithms like PPO, VPG, SARSA, A2C and ACKTR. The results show that our method is able to outperform state-of-the-art models by a huge margin including the original DQN based models. Finally, we test our model on a large and complex real building consisting of 91 rooms, with the possibility to move to any other room, hence giving 8281 actions. We use an attention based mechanism to deal with large action spaces. Our model achieves near optimal performance on the real world emergency environment.


page 1

page 12

page 13

page 17


Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles

A very successful model for simulating emergency evacuation is the socia...

Q-Cogni: An Integrated Causal Reinforcement Learning Framework

We present Q-Cogni, an algorithmically integrated causal reinforcement l...

A supervised hybrid quantum machine learning solution to the emergency escape routing problem

Managing the response to natural disasters effectively can considerably ...

Policy Optimization for Stochastic Shortest Path

Policy optimization is among the most popular and successful reinforceme...

Efficient Exploration in Constrained Environments with Goal-Oriented Reference Path

In this paper, we consider the problem of building learning agents that ...

Opinion shaping in social networks using reinforcement learning

In this paper, we study how to shape opinions in social networks when th...

Hybrid of representation learning and reinforcement learning for dynamic and complex robotic motion planning

Motion planning is the soul of robot decision making. Classical planning...

Please sign up or login with your details

Forgot password? Click here to reset