Solving Markov decision processes for network-level post-hazard recovery via simulation optimization and rollout

by   Yugandhar Sarkale, et al.
Colorado State University

Computation of optimal recovery decisions for community resilience assurance post-hazard is a combinatorial decision-making problem under uncertainty. It involves solving a large-scale optimization problem, which is significantly aggravated by the introduction of uncertainty. In this paper, we draw upon established tools from multiple research communities to provide an effective solution to this challenging problem. We provide a stochastic model of damage to the water network (WN) within a testbed community following a severe earthquake and compute near-optimal recovery actions for restoration of the water network. We formulate this stochastic decision-making problem as a Markov Decision Process (MDP), and solve it using a popular class of heuristic algorithms known as rollout. A simulation-based representation of MDPs is utilized in conjunction with rollout and the Optimal Computing Budget Allocation (OCBA) algorithm to address the resulting stochastic simulation optimization problem. Our method employs non-myopic planning with efficient use of simulation budget. We show, through simulation results, that rollout fused with OCBA performs competitively with respect to rollout with total equal allocation (TEA) at a meagre simulation budget of 5-10 which is a crucial step towards addressing large-scale community recovery problems following natural disasters.


Near-optimal planning using approximate dynamic programming to enhance post-hazard community resilience management

The lack of a comprehensive decision-making approach at the community-le...

Decision Automation for Electric Power Network Recovery

Critical infrastructure systems such as electric power networks, water n...

An Approximate Dynamic Programming Approach to Community Recovery Management (Extended Abstract)

The functioning of interdependent civil infrastructure systems in the af...

Budget Allocation in Binary Opinion Dynamics

In this article we study the allocation of a budget to promote an opinio...

An approximate dynamic programming approach to food security of communities following hazards

Food security can be threatened by extreme natural hazard events for hou...

Multiple Plans are Better than One: Diverse Stochastic Planning

In planning problems, it is often challenging to fully model the desired...

Please sign up or login with your details

Forgot password? Click here to reset