ASSED -- A Framework for Identifying Physical Events through Adaptive Social Sensor Data Filtering

09/17/2019
by   Abhijit Suprem, et al.
0

Physical event detection has long been the domain of static event processors operating on numeric sensor data. This works well for large scale strong-signal events such as hurricanes, and important classes of events such as earthquakes. However, for a variety of domains there is insufficient sensor coverage, e.g., landslides, wildfires, and flooding. Social networks have provided massive volume of data from billions of users, but data from these generic social sensors contain much more noise than physical sensors. One of the most difficult challenges presented by social sensors is concept drift, where the terms associated with a phenomenon evolve and change over time, rendering static machine learning (ML) classifiers less effective. To address this problem, we develop the ASSED (Adaptive Social Sensor Event Detection) framework with an ML-based event processing engine and show how it can perform simple and complex physical event detection on strong- and weak-signal with low-latency, high scalability, and accurate coverage. Specifically, ASSED is a framework to support continuous filter generation and updates with machine learning using streaming data from high-confidence sources (physical and annotated sensors) and social networks. We build ASSED to support procedures for integrating high-confidence sources into social sensor event detection to generate high-quality filters and to perform dynamic filter selection by tracking its own performance. We demonstrate ASSED capabilities through a landslide detection application that detects almost 350% more landslides compared to static approaches. More importantly, ASSED automates the handling of concept drift: four years after initial data collection and classifier training, ASSED achieves event detection accuracy of 0.988 (without expert manual intervention), compared to 0.762 for static approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/17/2019

Concept Drift Adaptive Physical Event Detection for Social Media Streams

Event detection has long been the domain of physical sensors operating i...
research
11/21/2019

Event Detection in Noisy Streaming Data with Combination of Corroborative and Probabilistic Sources

Global physical event detection has traditionally relied on dense covera...
research
03/15/2018

Event Correlation and Forecasting over Multivariate Streaming Sensor Data

Event management in sensor networks is a multidisciplinary field involvi...
research
05/13/2022

Detecting Rumours with Latency Guarantees using Massive Streaming Data

Today's social networks continuously generate massive streams of data, w...
research
01/23/2020

EventMapper: Detecting Real-World Physical Events Using Corroborative and Probabilistic Sources

The ubiquity of social media makes it a rich source for physical event d...
research
06/21/2023

Machine Learning Based Compensation for Inconsistencies in Knitted Force Sensors

Knitted sensors frequently suffer from inconsistencies due to innate eff...
research
12/22/2017

Virtual Sensor Modelling using Neural Networks with Coefficient-based Adaptive Weights and Biases Search Algorithm for Diesel Engines

With the explosion in the field of Big Data and introduction of more str...

Please sign up or login with your details

Forgot password? Click here to reset