A Multi-step and Resilient Predictive Q-learning Algorithm for IoT with Human Operators in the Loop: A Case Study in Water Supply Networks

by   Maria Grammatopoulou, et al.

We consider the problem of recommending resilient and predictive actions for an IoT network in the presence of faulty components, considering the presence of human operators manipulating the information of the environment the agent sees for containment purposes. The IoT network is formulated as a directed graph with a known topology whose objective is to maintain a constant and resilient flow between a source and a destination node. The optimal route through this network is evaluated via a predictive and resilient Q-learning algorithm which takes into account historical data about irregular operation, due to faults, as well as the feedback from the human operators that are considered to have extra information about the status of the network concerning locations likely to be targeted by attacks. To showcase our method, we utilize anonymized data from Arlington County, Virginia, to compute predictive and resilient scheduling policies for a smart water supply system, while avoiding (i) all the locations indicated to be attacked according to human operators (ii) as many as possible neighborhoods detected to have leaks or other faults. This method incorporates both the adaptability of the human and the computation capability of the machine to achieve optimal implementation containment and recovery actions in water distribution.


page 1

page 6

page 7


SAM-kNN Regressor for Online Learning in Water Distribution Networks

Water distribution networks are a key component of modern infrastructure...

Dynamic Resilient Network Games with Applications to Multi-Agent Consensus

A cyber security problem in a networked system formulated as a resilient...

Designing Adversarially Resilient Classifiers using Resilient Feature Engineering

We provide a methodology, resilient feature engineering, for creating ad...

Estimating irregular water demands with physics-informed machine learning to inform leakage detection

Leakages in drinking water distribution networks pose significant challe...

Scenario-based model predictive control of water reservoir systems

The optimal operation of water reservoir systems is a challenging task i...

An Architectural Design Decision Model for Resilient IoT Application

The Internet of Things is a paradigm that refers to the ubiquitous prese...

RADAMS: Resilient and Adaptive Alert and Attention Management Strategy against Informational Denial-of-Service (IDoS) Attacks

Attacks exploiting human attentional vulnerability have posed severe thr...

Please sign up or login with your details

Forgot password? Click here to reset