A Scalable and Dependable Data Analytics Platform for Water Infrastructure Monitoring

by   Felix Lorenz, et al.

With weather becoming more extreme both in terms of longer dry periods and more severe rain events, municipal water networks are increasingly under pressure. The effects include damages to the pipes, flash floods on the streets and combined sewer overflows. Retrofitting underground infrastructure is very expensive, thus water infrastructure operators are increasingly looking to deploy IoT solutions that promise to alleviate the problems at a fraction of the cost. In this paper, we report on preliminary results from an ongoing joint research project, specifically on the design and evaluation of its data analytics platform. The overall system consists of energy-efficient sensor nodes that send their observations to a stream processing engine, which analyzes and enriches the data and transmits the results to a GIS-based frontend. As the proposed solution is designed to monitor large and critical infrastructures of cities, several non-functional requirements such as scalability, responsiveness and dependability are factored into the system architecture. We present a scalable stream processing platform and its integration with the other components, as well as the algorithms used for data processing. We discuss significant challenges and design decisions, introduce an efficient data enrichment procedure and present empirical results to validate the compliance with the target requirements. The entire code for deploying our platform and running the data enrichment jobs is made publicly available with this paper.


RT-DAP: A Real-Time Data Analytics Platform for Large-scale Industrial Process Monitoring and Control

In most process control systems nowadays, process measurements are perio...

Towards Interactive, Adaptive and Result-aware Big Data Analytics

As data volumes grow across applications, analytics of large amounts of ...

OODIDA: On-board/Off-board Distributed Data Analytics for Connected Vehicles

Connected vehicles may produce gigabytes of data per hour, which makes c...

Dependable IoT Data Stream Processing for Monitoring and Control of Urban Infrastructures

The Internet of Things describes a network of physical devices interacti...

WaCoDiS: Automated Earth Observation Data Processing within an Event-Driven Architecture for Water Monitoring

To ensure an efficient and environmentally friendly water resource manag...

RHEEMix in the Data Jungle -- A Cross-Platform Query Optimizer --

In pursuit of efficient and scalable data analytics, the insight that "o...

An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics

Near-sensor data analytics is a promising direction for IoT endpoints, a...

Please sign up or login with your details

Forgot password? Click here to reset