The EpiBench Platform to Propel AI/ML-based Epidemic Forecasting: A Prototype Demonstration Reaching Human Expert-level Performance

02/04/2021
by   Ajitesh Srivastava, et al.
0

During the COVID-19 pandemic, a significant effort has gone into developing ML-driven epidemic forecasting techniques. However, benchmarks do not exist to claim if a new AI/ML technique is better than the existing ones. The "covid-forecast-hub" is a collection of more than 30 teams, including us, that submit their forecasts weekly to the CDC. It is not possible to declare whether one method is better than the other using those forecasts because each team's submission may correspond to different techniques over the period and involve human interventions as the teams are continuously changing/tuning their approach. Such forecasts may be considered "human-expert" forecasts and do not qualify as AI/ML approaches, although they can be used as an indicator of human expert performance. We are interested in supporting AI/ML research in epidemic forecasting which can lead to scalable forecasting without human intervention. Which modeling technique, learning strategy, and data pre-processing technique work well for epidemic forecasting is still an open problem. To help advance the state-of-the-art AI/ML applied to epidemiology, a benchmark with a collection of performance points is needed and the current "state-of-the-art" techniques need to be identified. We propose EpiBench a platform consisting of community-driven benchmarks for AI/ML applied to epidemic forecasting to standardize the challenge with a uniform evaluation protocol. In this paper, we introduce a prototype of EpiBench which is currently running and accepting submissions for the task of forecasting COVID-19 cases and deaths in the US states and We demonstrate that we can utilize the prototype to develop an ensemble relying on fully automated epidemic forecasts (no human intervention) that reaches human-expert level ensemble currently being used by the CDC.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/23/2020

Learning to Forecast and Forecasting to Learn from the COVID-19 Pandemic

Accurate forecasts of COVID-19 is central to resource management and bui...
research
06/03/2021

SIMLR: Machine Learning inside the SIR model for COVID-19 Forecasting

Accurate forecasts of the number of newly infected people during an epid...
research
02/20/2022

Chimeric forecasting: combining probabilistic predictions from computational models and human judgment

Forecasts of the trajectory of an infectious agent can help guide public...
research
09/11/2018

Forecasting Based on Surveillance Data

Forecasting the future course of epidemics has always been one of the ma...
research
06/11/2020

The Limits to Learning an SIR Process: Granular Forecasting for Covid-19

A multitude of forecasting efforts have arisen to support management of ...
research
02/24/2022

Comparison of Combination Methods to Create Calibrated Ensemble Forecasts for Seasonal Influenza in the U.S

The characteristics of influenza seasons varies substantially from year ...
research
09/08/2022

Shape-based Evaluation of Epidemic Forecasts

Infectious disease forecasting for ongoing epidemics has been traditiona...

Please sign up or login with your details

Forgot password? Click here to reset