Enabling real-time multi-messenger astrophysics discoveries with deep learning

by   E. A. Huerta, et al.

Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics.



There are no comments yet.


page 1

page 6


Deep Learning for Multi-Messenger Astrophysics: A Gateway for Discovery in the Big Data Era

This report provides an overview of recent work that harnesses the Big D...

Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation with LIGO Data

The recent Nobel-prize-winning detections of gravitational waves from me...

Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data

The recent Nobel-prize-winning detections of gravitational waves from me...

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

We live in momentous times. The science community is empowered with an a...

Machine Learning in Cyber-Security - Problems, Challenges and Data Sets

We present cyber-security problems of high importance. We show that in o...

A Graph-based Method for Session-based Recommendations

We present a graph-based approach for the data management tasks and the ...

AFEL-REC: A Recommender System for Providing Learning Resource Recommendations in Social Learning Environments

In this paper, we present preliminary results of AFEL-REC, a recommender...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.


  • [1] Abbott, B. P. et al. GWTC-1: A Gravitational-Wave Transient Catalog of Compact Binary Mergers Observed by LIGO and Virgo during the First and Second Observing Runs. Phys. Rev. X9, 031040 (2019). 1811.12907.
  • [2] Arnett, W. D., Bahcall, J. N., Kirshner, R. P. & Woosley, S. E. Supernova 1987a. Annual Review of Astronomy and Astrophysics 27, 629–700 (1989).
  • [3] Abbott, B. P. et al. GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral. Physical Review Letters 119, 161101 (2017). 1710.05832.
  • [4] Abbott, B. P. et al. Estimating the Contribution of Dynamical Ejecta in the Kilonova Associated with GW170817. Astrophys. J. 850, L39 (2017). 1710.05836.
  • [5] IceCube Collaboration. Neutrino emission from the direction of the blazar txs 0506+056 prior to the icecube-170922a alert. Science 361, 147–151 (2018).
  • [6] Large Synoptic Survey Telescope. LSST System and Survey Key Numbers. https://www.lsst.org/scientists/keynumbers.
  • [7] Collaboration, L. S., Abell, P. A. et al. LSST Science Book, Version 2.0 (2009). 0912.0201.
  • [8] Robertson, B. E. et al. Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope. Nature Rev. Phys. 1, 450–462 (2019).
  • [9] Owen, B. J. & Sathyaprakash, B. S. Matched filtering of gravitational waves from inspiraling compact binaries: Computational cost and template placement. Phys. Rev. D  60, 022002 (1999). gr-qc/9808076.
  • [10] Huerta, E. A. et al. Complete waveform model for compact binaries on eccentric orbits. Phys. Rev. D  95, 024038 (2017). 1609.05933.
  • [11] Harry, I., Privitera, S., Bohé, A. & Buonanno, A. Searching for gravitational waves from compact binaries with precessing spins. Phys. Rev. D  94, 024012 (2016). 1603.02444.
  • [12] Huerta, E. A. et al. BOSS-LDG: A Novel Computational Framework that Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery. In Proceedings, 13th International Conference on e-Science: Auckland, New Zealand, October 24-27, 2017, 335–344 (2017). 1709.08767.
  • [13] Huerta, E. A., Haas, R., Jha, S., Neubauer, M. & Katz, D. S. Supporting High-Performance and High-Throughput Computing for Experimental Science. Comput. Softw. Big Sci. 3, 5 (2019). 1810.03056.
  • [14] Weitzel, D. et al. Data Access for LIGO on the OSG. arXiv e-prints arXiv:1705.06202 (2017). 1705.06202.
  • [15] Abbott, B. P. et al. Observing gravitational-wave transient GW150914 with minimal assumptions. Phys. Rev. D  93, 122004 (2016). 1602.03843.
  • [16] Jones, P. W., Osipov, A. & Rokhlin, V. Randomized approximate nearest neighbors algorithm. Proceedings of the National Academy of Sciences 108, 15679–15686 (2011).
  • [17] Liang, S., Liu, Y., Wang, C. & Jian, L. Design and evaluation of a parallel k-nearest neighbor algorithm on cuda-enabled gpu. In 2010 IEEE 2nd Symposium on Web Society, 53–60 (2010).
  • [18] Asch, M. et al. Big data and extreme-scale computing: Pathways to convergence-toward a shaping strategy for a future software and data ecosystem for scientific inquiry. The International Journal of High Performance Computing Applications 32, 435–479 (2018).
  • [19] National Academies of Sciences, Engineering, and Medicine. Future Directions for NSF Advanced Computing Infrastructure to Support U.S. Science and Engineering in 2017-2020 (The National Academies Press, Washington, DC, 2016).
  • [20] Metzger, B. D. & Berger, E. What is the Most Promising Electromagnetic Counterpart of a Neutron Star Binary Merger? Astrophys. J. 746, 48 (2012). 1108.6056.
  • [21] Siegel, D. M. & Metzger, B. D. Three-dimensional grmhd simulations of neutrino-cooled accretion disks from neutron star mergers. The Astrophysical Journal 858, 52 (2018).
  • [22] Abbott, B. P. et al. Prospects for Observing and Localizing Gravitational-Wave Transients with Advanced LIGO and Advanced Virgo. Living Reviews in Relativity 19 (2016). 1304.0670.
  • [23] Drout, M. R. et al. Light curves of the neutron star merger GW170817/SSS17a: Implications for r-process nucleosynthesis. Science 358, 1570–1574 (2017). 1710.05443.
  • [24] Mooley, K. P. et al. A mildly relativistic wide-angle outflow in the neutron-star merger event GW170817. Nature 554, 207–210 (2018). 1711.11573.
  • [25] Andreoni, I. et al. Mary, a Pipeline to Aid Discovery of Optical Transients. Publications of the Astronomical Society of Australia 34, e037 (2017). 1708.04629.
  • [26] Sedaghat, N. & Mahabal, A. Effective image differencing with convolutional neural networks for real-time transient hunting. MNRAS  476, 5365–5376 (2018). 1710.01422.
  • [27] Kessler, R. et al. Results from the Supernova Photometric Classification Challenge. Publications of the Astronomical Society of the Pacific  122, 1415 (2010). 1008.1024.
  • [28] Jones, D. O. et al. Measuring Dark Energy Properties with Photometrically Classified Pan-STARRS Supernovae. II. Cosmological Parameters. Astrophys. J. 857, 51 (2018). 1710.00846.
  • [29] Scolnic, D. et al. How Many Kilonovae Can Be Found in Past, Present, and Future Survey Data Sets? Astrophys. J. Lett  852, L3 (2018). 1710.05845.
  • [30] Setzer, C. N. et al. Serendipitous Discoveries of Kilonovae in the LSST Main Survey: Maximising Detections of Sub-Threshold Gravitational Wave Events. arXiv e-prints arXiv:1812.10492 (2018). 1812.10492.
  • [31] Schutz, B. F. Determining the Hubble constant from gravitational wave observations. Nature 323, 310 (1986).
  • [32] The DES Collaboration, the LIGO Scientific Collaboration & the Virgo Collaboration. First measurement of the Hubble constant from a dark standard siren using the Dark Energy Survey galaxies and the LIGO/Virgo binary-black-hole merger GW170814. arXiv e-prints arXiv:1901.01540 (2019). 1901.01540.
  • [33] Cowperthwaite, P. S. et al. The electromagnetic counterpart of the binary neutron star merger ligo/virgo gw170817. ii. uv, optical, and near-infrared light curves and comparison to kilonova models. The Astrophysical Journal Letters 848, L17 (2017).
  • [34] Abbott, B. P. et al. A gravitational-wave standard siren measurement of the Hubble constant. Nature 551, 85–88 (2017).
  • [35] Fishbach, M. et al. A standard siren measurement of the Hubble constant from GW170817 without the electromagnetic counterpart. ArXiv e-prints arXiv:1807.05667 (2018). 1807.05667.
  • [36] Khan, A. et al. Deep learning at scale for the construction of galaxy catalogs in the Dark Energy Survey. Physics Letters B 795, 248–258 (2019). 1812.02183.
  • [37] Domínguez Sánchez, H., Huertas-Company, M., Bernardi, M., Tuccillo, D. & Fischer, J. L. Improving galaxy morphologies for SDSS with Deep Learning. MNRAS  476, 3661–3676 (2018).
  • [38] Eisenstein, D. J. et al. SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way, and Extra-Solar Planetary Systems. The Astronomical Journal 142, 72 (2011). 1101.1529.
  • [39] Dark Energy Survey Collaboration et al. The Dark Energy Survey: more than dark energy - an overview. MNRAS  460, 1270–1299 (2016). 1601.00329.
  • [40] Riess, A. G., Casertano, S., Yuan, W., Macri, L. M. & Scolnic, D. Large Magellanic Cloud Cepheid Standards Provide a 1% Foundation for the Determination of the Hubble Constant and Stronger Evidence for Physics beyond CDM. Astrophys. J. 876, 85 (2019). 1903.07603.
  • [41] Aghanim, N. et al. Planck 2018 results. VI. Cosmological parameters (2018). 1807.06209.
  • [42] Poulin, V., Smith, T. L., Karwal, T. & Kamionkowski, M. Early dark energy can resolve the hubble tension. Phys. Rev. Lett. 122, 221301 (2019).
  • [43] Freedman, W. L. Cosmology at a crossroads. Nature Astronomy 1, 0121 (2017).
  • [44] Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
  • [45] George, D. & Huerta, E. A. Deep neural networks to enable real-time multimessenger astrophysics. Phys. Rev. D  97, 044039 (2018). 1701.00008.
  • [46] George, D. & Huerta, E. A. Deep Learning for real-time gravitational wave detection and parameter estimation: Results with Advanced LIGO data. Physics Letters B 778, 64–70 (2018). 1711.03121.
  • [47] Rebei, A. et al. Fusing numerical relativity and deep learning to detect higher-order multipole waveforms from eccentric binary black hole mergers. Phys. Rev. D  100, 044025 (2019). 1807.09787.
  • [48] Wei, W. & Huerta, E. A. Gravitational Wave Denoising of Binary Black Hole Mergers with Deep Learning. Physics Letters B 800, 135081 (2020). 1901.00869.
  • [49] Shen, H., George, D., Huerta, E. A. & Zhao, Z. Denoising gravitational waves with enhanced deep recurrent denoising auto-encoders. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3237–3241 (2019).
  • [50] Shen, H., George, D., Huerta, E. A. & Zhao, Z.

    Denoising Gravitational Waves using Deep Learning with Recurrent Denoising Autoencoders.

    ArXiv e-prints (2017). 1711.09919.
  • [51] George, D., Shen, H. & Huerta, E. A.

    Classification and unsupervised clustering of LIGO data with Deep Transfer Learning.

    Phys. Rev. D  97, 101501 (2018). 1706.07446.
  • [52] Chua, A. J. K., Galley, C. R. & Vallisneri, M.

    Reduced-order modeling with artificial neurons for gravitational-wave inference.

    Phys. Rev. Lett. 122, 211101 (2019). 1811.05491.
  • [53] Gabbard, H., Williams, M., Hayes, F. & Messenger, C. Matching Matched Filtering with Deep Networks for Gravitational-Wave Astronomy. Physical Review Letters 120, 141103 (2018). 1712.06041.
  • [54] Nakano, H. et al. Comparison of various methods to extract ringdown frequency from gravitational wave data. Phys. Rev. D99, 124032 (2019). 1811.06443.
  • [55] Dreissigacker, C., Sharma, R., Messenger, C., Zhao, R. & Prix, R. Deep-Learning Continuous Gravitational Waves. Phys. Rev. D100, 044009 (2019). 1904.13291.
  • [56] Shen, H., Huerta, E. A. & Zhao, Z. Deep Learning at Scale for Gravitational Wave Parameter Estimation of Binary Black Hole Mergers. arXiv e-prints arXiv:1903.01998 (2019). 1903.01998.
  • [57] Springenberg, J. T., Klein, A., Falkner, S. & Hutter, F. Bayesian optimization with robust bayesian neural networks. In Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, 4134–4142 (Curran Associates, Inc., 2016).
  • [58] Burrows, A., Hayes, J. & Fryxell, B. A. On the nature of core collapse supernova explosions. Astrophys. J. 450, 830 (1995). astro-ph/9506061.
  • [59] Burrows, A., Radice, D. & Vartanyan, D. Three-dimensional supernova explosion simulations of 9-, 10-, 11-, 12-, and 13-M stars. Mon. Not. Roy. Astron. Soc. 485, 3153–3168 (2019). 1902.00547.
  • [60] Radice, D., Morozova, V., Burrows, A., Vartanyan, D. & Nagakura, H. Characterizing the Gravitational Wave Signal from Core-Collapse Supernovae. Astrophys. J. 876, L9 (2019). 1812.07703.
  • [61] Mösta, P. et al. R-process Nucleosynthesis from Three-Dimensional Magnetorotational Core-Collapse Supernovae. Astrophys. J. 864, 171 (2018). 1712.09370.
  • [62] Janka, H. T., Melson, T. & Summa, A. Physics of Core-Collapse Supernovae in Three Dimensions: a Sneak Preview. Ann. Rev. Nucl. Part. Sci. 66, 341–375 (2016). 1602.05576.
  • [63] Woosley, S. & Janka, T. The physics of core-collapse supernovae. Nature Phys. 1, 147 (2005). astro-ph/0601261.
  • [64] Gossan, S. E. et al. Observing Gravitational Waves from Core-Collapse Supernovae in the Advanced Detector Era. Phys. Rev. D93, 042002 (2016). 1511.02836.
  • [65] Aurisano, A. et al. A Convolutional Neural Network Neutrino Event Classifier. JINST 11, P09001 (2016). 1604.01444.
  • [66] Choma, N. et al. Graph neural networks for icecube signal classification. In Wani, M., Sayed-Mouchaweh, M., Lughofer, E., Gama, J. & Kantardzic, M. (eds.) Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, 386–391 (Institute of Electrical and Electronics Engineers Inc., 2019).
  • [67] Hinderer, T. et al. Distinguishing the nature of comparable-mass neutron star binary systems with multimessenger observations: GW170817 case study. Phys. Rev. D100, 06321 (2019). 1808.03836.
  • [68] Kim, B. et al. Deep fluids: A generative network for parameterized fluid simulations. Computer Graphics Forum 38, 59–70 (2019).
  • [69] Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics 807, 155–166 (2016).
  • [70] Maulik, R., San, O., Rasheed, A. & Vedula, P. Subgrid modelling for two-dimensional turbulence using neural networks. Journal of Fluid Mechanics 858, 122–144 (2019).
  • [71] Viganò, D., Aguilera-Miret, R. & Palenzuela, C. Extension of the subgrid-scale gradient model for compressible magnetohydrodynamics turbulent instabilities. Physics of Fluids 31, 105102 (2019).
  • [72] Xie, C., Wang, J., Li, K. & Ma, C. Artificial neural network approach to large-eddy simulation of compressible isotropic turbulence. Phys. Rev. E 99, 053113 (2019).
  • [73] Berg, J. & Nyström, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317, 28–41 (2018).
  • [74] Weinan, E., Han, J. & Jentzen, A. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Communications in Mathematics and Statistics 5, 349–380 (2017).
  • [75] Duez, M. D. & Zlochower, Y. Numerical relativity of compact binaries in the 21st century. Reports on Progress in Physics 82, 016902 (2019). 1808.06011.
  • [76] Baiotti, L. & Rezzolla, L. Binary neutron star mergers: a review of einstein’s richest laboratory. Reports on Progress in Physics 80, 096901 (2017).
  • [77] Lippuner, J. & Roberts, L. F. Skynet: A modular nuclear reaction network library. The Astrophysical Journal Supplement Series 233, 18 (2017).
  • [78] Paschalidis, V., Ruiz, M. & Shapiro, S. L. Relativistic Simulations of Black Hole–neutron Star Coalescence: the jet Emerges. Astrophys. J. 806, L14 (2015). 1410.7392.
  • [79] Ruiz, M., Lang, R. N., Paschalidis, V. & Shapiro, S. L. Binary Neutron Star Mergers: a jet Engine for Short Gamma-ray Bursts. Astrophys. J. 824, L6 (2016). 1604.02455.
  • [80] Fernández, R. et al. Long-term GRMHD Simulations of Neutron Star Merger Accretion Disks: Implications for Electromagnetic Counterparts. ArXiv e-prints (2018). 1808.00461.
  • [81] Hossein Nouri, F. et al. Evolution of the magnetized, neutrino-cooled accretion disk in the aftermath of a black hole-neutron star binary merger. Phys. Rev. D 97, 083014 (2018).
  • [82] Radice, D. et al. Binary Neutron Star Mergers: Mass Ejection, Electromagnetic Counterparts, and Nucleosynthesis. Astrophys. J. 869, 130 (2018). 1809.11161.
  • [83] Kasen, D., Badnell, N. R. & Barnes, J. Opacities and Spectra of the r-process Ejecta from Neutron Star Mergers. ApJ 774, 25 (2013). 1303.5788.
  • [84] Berger, M. & Colella, P. Local adaptive mesh refinement for shock hydrodynamics. Journal of Computational Physics 82, 64 – 84 (1989).
  • [85] Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. arXiv preprint arXiv:1806.07366 (2018).
  • [86] Radice, D. et al. Neutrino-Driven Convection in Core-Collapse Supernovae: High-Resolution Simulations. Astrophys. J. 820, 76 (2016). 1510.05022.
  • [87] Giacomazzo, B., Zrake, J., Duffell, P., MacFadyen, A. I. & Perna, R. Producing Magnetar Magnetic Fields in the Merger of Binary Neutron Stars. Astrophys. J. 809, 39 (2015). 1410.0013.
  • [88] EuroHPC. EuroHPC: Leading the way in the European Supercomputing. https://eurohpc-ju.europa.eu/index.html. 2018.
  • [89] Huerta, E. A. et al. Boss-ldg: A novel computational framework that brings together blue waters, open science grid, shifter and the ligo data grid to accelerate gravitational wave discovery. In 2017 IEEE 13th International Conference on e-Science (e-Science), 335–344 (2017).
  • [90] Hotokezaka, K., Beniamini, P. & Piran, T. Neutron star mergers as sites of r-process nucleosynthesis and short gamma-ray bursts. International Journal of Modern Physics D 27, 1842005 (2018). 1801.01141.
  • [91] Löffler, F. et al. The Einstein Toolkit: a community computational infrastructure for relativistic astrophysics. Classical and Quantum Gravity 29, 115001 (2012). 1111.3344.
  • [92] Radice, D. & Rezzolla, L. THC: a new high-order finite-difference high-resolution shock-capturing code for special-relativistic hydrodynamics. Astron. Astrophys. 547, A26 (2012). 1206.6502.
  • [93] Arcavi, I. et al. Optical Follow-up of Gravitational-wave Events with Las Cumbres Observatory. Astrophys. J. 848, L33 (2017). 1710.05842.
  • [94] Coughlin, M. W. et al. Optimizing searches for electromagnetic counterparts of gravitational wave triggers. Mon. Not. Roy. Astron. Soc. 478, 692–702 (2018). 1803.02255.
  • [95] NED. California Institute of Technology NED gravitational wave follow-up service. https://ned.ipac.caltech.edu/gwf/overview. 2019.
  • [96] AMON. Pennsylvania State University Astrophysical multimessenger observatory network. https://www.amon.psu.edu/amon-system/. 2019.
  • [97] Marshall, P. et al. Science-Driven Optimization of the LSST Observing Strategy (2017). 1708.04058.
  • [98] W. M. Keck Observatory Scientific Strategic Plan. The W. M. Keck Observatory Scientific Strategic Plan. https://www.ucolick.org/home/about/governance/Keck_Scientific_Strategic_Plan_Final.pdf (2016). Online from July 2016.
  • [99] Cowperthwaite, P. S. et al. Astro 2020 Science White Paper: Joint Gravitational Wave and Electromagnetic Astronomy with LIGO and LSST in the 2020’s (2019). 1904.02718.
  • [100] Narayan, G. et al. Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream. Astrophys. J. Suppl. 236, 9 (2018). 1801.07323.
  • [101] Smith, K. W. et al. Lasair: The Transient Alert Broker for LSST:UK. Research Notes of the American Astronomical Society 3, 26 (2019).
  • [102] AEON Team. Astronomical event observatory network (2018). http://ast.noao.edu/data/aeon.
  • [103] The National Academies of Sciences, Engineering, and Medicine. The National Academies of Sciences, Engineering, and Medicine. The decadal survey on astronomy and astrophysics (astro2020. https://sites.nationalacademies.org/DEPS/astro2020/index.htm. 2019.
  • [104] Katz, D. S. et al. Community organizations: Changing the culture in which research software is developed and sustained. Comp. in Sci. & Eng. 21, 8–24 (2019).
  • [105] Elmer, P., Neubauer, M. & Sokoloff, M. D. Strategic Plan for a Scientific Software Innovation Institute (S2I2) for High Energy Physics. arXiv e-prints arXiv:1712.06592 (2017). 1712.06592.
  • [106] Albrecht, J. et al. A Roadmap for HEP Software and Computing R&D for the 2020s. arXiv e-prints arXiv:1712.06982 (2017). 1712.06982.
  • [107] Allen, G. et al. Multi-Messenger Astrophysics: Harnessing the Data Revolution. arXiv e-prints arXiv:1807.04780 (2018). 1807.04780.