Machine learning has a growing role in increasing health service access and efficiency, particularly in resource-constrained settings, making it a valuable tool for the global health community [Panch et al., 2018, Wahl et al., 2018]. Moreover, the COVID-19 pandemic [WHO, 2020] has underlined the importance of epidemiological modelling and computer simulation in informing the design and implementation of public health interventions at an unprecedented scale [Eubank et al., 2020]. For many endemic diseases (e.g., malaria), in-silico optimisation of multi-modal intervention portfolios—from mass vaccination to bed nets—is well established [Smith et al., 2017]. Analogous modelling for COVID-19 interventions, including social distancing [Ferguson et al., 2006], is mostly unexplored, yet subject to intense public interest [Mahase, 2020].
The adoption of health informatics in worldwide health systems (e.g., OpenMRS [Mamlin et al., 2006], mHealth [Adibi, 2015]) enables access to abundant patient-level and aggregated health data [Wahl et al., 2018]. This is fomenting the development of comprehensive modelling and simulation to support the design of health interventions and policies, and to guide decision-making in a variety of health system domains [Fone et al., 2003, Swanson et al., 2012]. For example, simulations have provided valuable insight to deal with public health problems such as tobacco consumption in New Zealand [Tobias et al., 2010], and diabetes and obesity in the US [Zhang et al., 2014]. They have been used to explore policy options such as those in maternal and antenatal care in Uganda [Semwanga et al., 2016], and applied to evaluate health reform scenarios such as predicting changes in access to primary care services in Portugal [Fialho et al., 2011]. Their applicability in informing the design of cancer screening programmes has been also discussed [Ronco and Baussano, 2017, Getaneh et al., 2019]. Recently, simulations have informed the response to the COVID-19 outbreak [Ferguson et al., 2020].
The process of informing health interventions and policies through simulations generally involves two steps:
- Model calibration
- Optimising decision-making
Identifying optimal multimodal intervention strategies and corresponding risks and uncertainties requires searching through potentially vast parameter spaces, which, due to the computational cost of running large simulators (e.g., in some epidemiological studies), usually cannot be exhaustively evaluated [Smith et al., 2012].
Despite their fundamental importance, model discrepancy and calibration of public-health simulators are frequently only informally addressed, or left undocumented [Stout et al., 2009, Punyacharoensin et al., 2011]. This may be partially explained by the fact that, while numerous methods for formal sensitivity and uncertainty analysis exist [Kennedy and O’Hagan, 2001], they in general do not scale to complex simulators with more than a few dozen parameters [Oakley and O’Hagan, 2004]. Similarly, evidence-based decision-making is usually optimised by comparing outcomes on a small number of hand-crafted scenarios and intervention strategies [Smith et al., 2012].
2 Epidemiology simulations and inference
Among the simplest mathematical epidemiology models are deterministic compartmental models that partition individuals in a population based on different stages of the disease333E.g., susceptible–infectious–recovered or SIR [Kermack and McKendrick, 1927, Allen, 1994, Brauer, 2008]. Advances in model construction, computing power, and novel insights into medical and socio-economic aspects have since stimulated the introduction of stochastic individual-based models 444Also referred to as agent-based model or multi-agent system. [Railsback and Grimm, 2019] to public health applications. These relatively complex and highly-parametrised models are implemented as simulator software that allow studying the global effects of self-organization and emergent properties arising from individual interactions at the local level.
In general, inference in individual-based simulators is usually doubly intractable, as both simulator likelihood and evidence cannot be evaluated efficiently. Likelihood-free methods, including ABC [Beaumont et al., 2002] have been proposed [Andrianakis et al., 2015, Csilléry et al., 2010], but suffer from exponential scaling of inference with data dimension, requiring domain experts to define low-dimensional summary statistics, which ultimately determine quality of inference.
Recent advances in machine learning have led to a new family of promising approaches to simulation-based inference [see Cranmer et al., 2020, for an overview]. In particular, we argue that probabilistic programming [see van de Meent et al., 2018, for an introduction] has a unique potential to standardise and automate model calibration and decision-making in individual-based simulators.
Probabilistic programming allows one to express probability models using computer code and perform statistical inference over the inputs and latent variables of the program, conditioned on data observations (or constraints). This is achieved by using special-purpose probabilistic programming languages (PPLs)[Goodman et al., 2012, Salvatier et al., 2016, Carpenter et al., 2017, Tran et al., 2018, Bingham et al., 2019], which augment a host language with features to express probabilities and Bayesian conditioning. PPLs separate model specification from inference, allowing flexible selection of appropriate inference engines (e.g., IS [Doucet and Johansen, 2009], MCMC [Brooks et al., 2011]).
Recent work made it possible to use pre-existing stochastic simulators as probabilistic programs, with minimal code modification to capture and redirect random number draws [Baydin et al., 2019], scaling up to very large simulators [Baydin et al., 2019], and particularly relevant to this work, with application to individual-based epidemiology simulators [Gram-Hansen et al., 2019b]. Within such a framework, one could, for instance, condition on desired health outcomes (e.g., ICU capacity not being exceeded in a pandemic), and derive detailed posterior distributions over all interactions defined by the simulator [Wood et al., 2020], providing insights on interventions effecting a desired outcome—with proper uncertainty quantification at all stages.
To further enhance the applicability of simulation-based inference in this domain we highlight several opportunities for further method development. Automated amortisation by surrogate methods [Gram-Hansen et al., 2019a, Naderiparizi et al., 2019, Munk et al., 2019], which aim to automatically identify and replace compute-intensive parts of a simulator through less expensive emulators, could be guided by the causal structure inherent to a simulator (Figure 1), such as many repeated, structurally identical stochastic time steps or multi-agent interactions that might be amenable to mean-field approximations [Yang et al., 2018]. In addition, pre-existing simulators could be turned into differentiable programs [Baydin et al., 2018] through automated source-to-source transformations, thus allowing for the use of gradient-based optimisation and inference methods, including Hamiltonian Monte Carlo [Neal, 2012]
. Last but not least, the unified interface specification afforded by probabilistic programming could allow simulators to also become amenable to other techniques from simulation-based inference and control, including dynamic programming and reinforcement learning[Levine, 2018, van de Meent et al., 2016, Igl et al., 2019].
To foster the development of a new standardised approach to model calibration and evidence-based decision-making in public health, we are working on instrumenting the existing CovidSim [Ferguson et al., 2020] and OpenMalaria [Smith et al., 2006] simulators with a probabilistic programming interface through the PyProb library.555https://github.com/pyprob/pyprob We will publicly release our code to provide out-of-the-box probabilistic programming inference over public health scenarios of interest in these two domains.
We expect the mentioned techniques to play a role in dealing with communicable (infectious) diseases, which already entailed a significant burden for health systems in developing countries [Murray et al., 2014] before the worldwide impact of COVID-19. However, they can also be applicable to non-communicable diseases, such as diabetes and cancer, which are recognised major causes of morbidity and mortality worldwide [Ansah et al., 2019, Kruk et al., 2018]. This will add to the already identified potential of machine learning in health policy [Ashrafian and Darzi, 2018] and improving health access, emphasising its value for global health in the efforts to achieve universal health coverage and sustainable development goals [Panch et al., 2018, Wahl et al., 2018].
- Mobile health: a technology road map. Vol. 5, Springer. Cited by: §1.
- Some discrete-time SI, SIR, and SIS epidemic models. Mathematical Biosciences 124 (1), pp. 83–105. Cited by: §2.
- Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda. PLOS Computational Biology 11 (1), pp. e1003968 (en). External Links: Cited by: item Model calibration, §2.
- Systems modelling as an approach for understanding and building consensus on non-communicable diseases (NCD) management in Cambodia. BMC Health Services Research 19 (1), pp. 1–10. Cited by: §2.
- Transforming health policy through machine learning. PLoS Medicine 15 (11). Cited by: §2.
- Efficient probabilistic inference in the quest for physics beyond the standard model. In Advances in Neural Information Processing Systems 33 (NeurIPS), External Links: Cited by: §2.
- Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research (JMLR) 18 (153), pp. 1–43. External Links: Cited by: §2.
- Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–24. Note: arXiv: 1907.03382Comment: 14 pages, 8 figures External Links: Cited by: §2.
- Approximate Bayesian computation in population genetics. Genetics 162 (4), pp. 2025–2035. Cited by: §2.
- Pyro: deep universal probabilistic programming. The Journal of Machine Learning Research 20 (1), pp. 973–978. Cited by: §2.
- Compartmental models in epidemiology. In Mathematical Epidemiology, pp. 19–79. Cited by: §2.
Handbook of markov chain monte carlo. CRC press. Cited by: §2.
- Learning about physical parameters: the importance of model discrepancy. Inverse Problems 30 (11), pp. 114007. Cited by: item Model calibration.
- Stan: a probabilistic programming language. Journal of Statistical Software 76 (1). Cited by: §2.
- The frontier of simulation-based inference. arXiv:1911.01429 [cs, stat]. Note: arXiv: 1911.01429Comment: 10 pages, 3 figures, proceedings for the Sackler Colloquia at the US National Academy of Sciences. v2: fixed typos. v3: clarified text, added references External Links: Cited by: §2.
- Approximate Bayesian computation (ABC) in practice. Trends in Ecology & Evolution 25 (7), pp. 410–418. Cited by: §2.
A tutorial on particle filtering and smoothing: fifteen years later.
Handbook of Nonlinear Filtering12 (656-704), pp. 3. Cited by: §2.
- Commentary on Ferguson, et al., “Impact of Non-pharmaceutical Interventions (NPIs) to Reduce COVID-19 Mortality and Healthcare Demand”. Bulletin of Mathematical Biology 82 (4). External Links: Cited by: §1.
- Report 9: impact of non-pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand. Cited by: §1, §2.
- Strategies for mitigating an influenza pandemic. Nature 442 (7101), pp. 448–452 (en). External Links: Cited by: §1.
- Using discrete event simulation to compare the performance of family health unit and primary health care centre organizational models in Portugal. BMC Health Services Research 11 (1), pp. 274. Cited by: §1.
- Systematic review of the use and value of computer simulation modelling in population health and health care delivery. Journal of Public Health 25 (4), pp. 325–335. External Links: Cited by: §1.
- The role of modelling in the policy decision making process for cancer screening: example of prostate specific antigen screening. Public Health Research & Practice 29 (2). Cited by: §1.
- Church: a language for generative models. arXiv preprint arXiv:1206.3255. Cited by: §2.
- Efficient Bayesian inference for nested simulators. In Second Symposium on Advances in Approximate Bayesian Inference (AABI), Vancouver, Canada, 8 December 2019, Cited by: §2.
- Hijacking malaria simulators with probabilistic programming. In ICML Workshop on AI for Social Good, Thirty-sixth International Conference on Machine Learning (ICML 2019), Long Beach, CA, US, Cited by: §2.
- Multitask soft option learning. arXiv preprint arXiv:1904.01033. Cited by: §2.
- Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63 (3), pp. 425–464 (en). Note: _eprint: https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/1467-9868.00294 External Links: Cited by: §1.
- A contribution to the mathematical theory of epidemics. Proceedings of the Royal Society of London. Series A, Containing papers of a mathematical and physical character 115 (772), pp. 700–721. Cited by: §2.
- High-quality health systems in the sustainable development goals era: time for a revolution. The Lancet Global Health 6 (11), pp. e1196–e1252. Cited by: §2.
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review. arXiv:1805.00909 [cs, stat]. Note: arXiv: 1805.00909 External Links: Cited by: §2.
- Covid-19: public health experts demand evidence behind UK’s short self-isolation advice. BMJ 369. External Links: Cited by: §1.
Cooking up an open source EMR for developing countries: OpenMRS–a recipe for successful collaboration. In AMIA Annual Symposium Proceedings, Vol. 2006, pp. 529. Cited by: §1.
- Deep Probabilistic Surrogate Networks for Universal Simulator Approximation. arXiv:1910.11950 [cs, stat]. Note: arXiv: 1910.11950 External Links: Cited by: §2.
- Global, regional, and national incidence and mortality for HIV, tuberculosis, and malaria during 1990–2013: a systematic analysis for the global burden of disease study 2013. The Lancet 384 (9947), pp. 1005–1070. Cited by: §2.
- Amortized rejection sampling in universal probabilistic programming. arXiv preprint arXiv:1910.09056. Cited by: §2.
- MCMC using Hamiltonian dynamics. arXiv:1206.1901 [physics, stat]. Note: arXiv: 1206.1901 External Links: Cited by: §2.
- Probabilistic sensitivity analysis of complex models: a Bayesian approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 66 (3), pp. 751–769. Cited by: §1.
- Artificial intelligence, machine learning and health systems. Journal of Global Health 8 (2). Cited by: §1, §2.
- Mathematical models for the study of HIV spread and control amongst men who have sex with men. European Journal of Epidemiology 26 (9), pp. 695. Cited by: §1.
- Agent-based and individual-based modeling: a practical introduction. Princeton University Press. Cited by: §2.
- Causal system modelling of cervical cancer screening. The Lancet Public Health 2 (2), pp. e61–e62. Cited by: §1.
- Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2, pp. e55. Cited by: §2.
- Applying a system dynamics modelling approach to explore policy options for improving neonatal health in Uganda. Health Research Policy and Systems 14 (1), pp. 35. Cited by: §1.
- Malaria Modeling in the Era of Eradication. Cold Spring Harbor Perspectives in Medicine 7 (4), pp. a025460 (en). Note: Publisher: Cold Spring Harbor Laboratory Press External Links: Cited by: §1.
- Mathematical modeling of the impact of malaria vaccines on the clinical epidemiology and natural history of plasmodium falciparum malaria: overview. The American Journal of Tropical Medicine and Hygiene 75 (2_suppl), pp. 1–10. Cited by: §2.
- Ensemble Modeling of the Likely Public Health Impact of a Pre-Erythrocytic Malaria Vaccine. PLOS Medicine 9 (1), pp. e1001157 (en). External Links: Cited by: item Optimising decision-making, §1.
- Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics 27 (7), pp. 533–545. Cited by: §1.
- Rethinking health systems strengthening: key systems thinking tools and strategies for transformational change. Health Policy and Planning 27 (suppl_4), pp. iv54–iv61. Cited by: §1.
- Application of a system dynamics model to inform investment in smoking cessation services in New Zealand. American Journal of Public Health 100 (7), pp. 1274–1281. Cited by: §1.
- Simple, distributed, and accelerated probabilistic programming. In Neural Information Processing Systems, Cited by: §2.
- Black-Box Policy Search with Probabilistic Programs. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pp. 1195–1204. Cited by: §2.
- An introduction to probabilistic programming. arXiv preprint arXiv:1809.10756. Cited by: §2.
- Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings?. BMJ Global Health 3 (4). External Links: Cited by: §1, §1, §2.
- Coronavirus. (en). Note: Library Catalog: www.who.int External Links: Cited by: §1.
- Planning as Inference in Epidemiological Models. arXiv:2003.13221 [cs, q-bio, stat]. Note: arXiv: 2003.13221Comment: minor typos corrected External Links: Cited by: §2.
- Mean Field Multi-Agent Reinforcement Learning. arXiv:1802.05438 [cs]. Note: arXiv: 1802.05438Comment: ICML 2018 (Full paper + Long talk) External Links: Cited by: §2.
- Impact of different policies on unhealthy dietary behaviors in an urban adult population: an agent-based simulation model. American Journal of Public Health 104 (7), pp. 1217–1222. Cited by: §1.