Solar flares and the often-associated coronal mass ejections (CMEs) highly impact the near-Earth space environment [1, 2]. They have the potential to cause catastrophic damage to technology infrastructure . According to the U.S. National Space Weather Strategy, released by the Space Weather Prediction Center, it is a challenging task to correctly predict solar flares and CMEs. Recent efforts led by the United States and its partners resulted in substantial progress toward monitoring, prediction, and mitigation plans, but much more effort is still needed.
Researches have indicated that the magnetic free energy stored in the corona, quickly discharged by magnetic reconnection, powers solar flares and CMEs  . The process of building the coronal free energy is controlled by the structural evolution of the magnetic field on the photosphere where plasma dominates the process. Observing and measuring the structure and evolution of the photospheric magnetic field can provide valuable information and clues to the triggering mechanisms of flares and CMEs. There are many physical properties or parameters, as we will discuss later in the paper, that characterize the static photospheric magnetic field, such as integrated Lorentz force, magnetic helicity injection, unsigned magnetic flux, vertical electric currents, magnetic shear and gradient, and magnetic energy dissipation.
Researchers spent significant efforts attempting to understand the physical relationship between flare productivity and non-potentiality of active regions (ARs) as specified by the physical parameters. This led researchers to use different methods to predict flares that are not based on physical models, but rather based on statistical modeling and machine learning . Machine learning gives computer programs the ability to learn from data and progressively improve performance. It uses input data, also called training data, and learns hidden insights in the training data to build a predictive model that will be used later to make predictions on unseen test data.
In our previous work 
, we reported the results of solar flare prediction using the random forests (RF) algorithm. We constructed a database of solar flare events using the physical parameters provided by the Space-weather HMI Active Region Patches (SHARP), and categorized solar flares into four different classes, namely B, C, M, X, based on the X-ray flare catalogs available at the National Centers for Environmental Information (NCEI). Flares in the B class have the smallest magnitude while flares in the X class have the largest magnitude. We used the RF algorithm and the physical parameters or features to perform multi-class classification of solar flares, predicting the occurrence of a certain class of flares in a given active region (AR) within 24 hours. Our experimental results demonstrated the good performance of the RF algorithm.
In this paper, we extend our previous work in 
by considering two additional multi-class classification algorithms: multilayer perceptrons (MLP) and extreme learning machines (ELM). We implement these algorithms into a machine-learning-as-a-service (MLaaS) framework, called DeepSun, which allows scientists to perform multi-class flare prediction on the Internet. Specifically, our work here makes two contributions.
We develop an ensemble method for multi-class flare prediction that performs better than the existing machine learning algorithms including RF, MLP and ELM according to our experimental study.
We design and implement DeepSun, which is the first MLaaS system of its kind for solar flare prediction.
The rest of this paper is organized as follows. Section II describes the data and the SHARP predictive parameters used in this study. Section III describes the machine learning algorithms employed by DeepSun. Section IV evaluates the performance of these machine learning algorithms. Section V details the design and implementation of the DeepSun framework. Section VI surveys related work and compares DeepSun with existing services computing systems. Section VII concludes the paper and points out some directions for future research.
Ii Data and Physical Parameters
In 2012, SHARP data were released. The main goal of the SHARP data was to facilitate AR (active region) event forecasting . These data are available in the Joint Science Operations Center (JSOC) (http://jsoc.stanford.edu/
) as hmi.sharp series which include magnetic measures and parameters for many ARs. In 2014, another data series, cgem.Lorentz, were produced based on the SHARP data. This series include the Lorentz force estimations. The main goal of this series was to help diagnose the dynamic process of ARs. Bobraet al. 
considered 25 physical parameters in the SHARP datasets that characterize the AR magnetic field properties. The authors used a univariate feature selection method to score the 25 parameters, and suggested that the top 13 out of the 25 parameters be used as predictors for flare activity. TableI summarizes these 13 important parameters and their descriptions. More details about the 13 physical parameters can be found in .
|ABSNJZH||Absolute value of the net current helicity|
|AREA_ACR||Area of strong field pixels in the active region|
|EPSZ||Sum of z-component of normalized Lorentz force|
|MEANPOT||Mean photospheric magnetic free energy|
|R_VALUE||Sum of flux near polarity inversion line|
|SAVNCPP||Sum of the modulus of the net current per polarity|
|SHRGT45||Fraction of area with shear 45|
|TOTBSQ||Total magnitude of Lorentz force|
|TOTFZ||Sum of z-component of Lorentz force|
|TOTPOT||Total photospheric magnetic free energy density|
|TOTUSJH||Total unsigned current helicity|
|TOTUSJZ||Total unsigned vertical current|
|USFLUX||Total unsigned flux|
We constructed a database based on the SHARP parameters extracted from the solar images that are available at the Joint Science Operations Center (JSOC) and the X-ray flare catalogs provided by the National Centers for Environmental Information (NCEI) . We considered the period between May 2010 and December 2016. There are 845 flares in this period, among which 128 flares are of class B, 552 flares are of class C, 142 flares are of class M, and 23 flares are of class X. These 845 flares come from 472 active regions (ARs). The duration of a flare ranges from several minutes to hours. The duration of an AR ranges from several minutes to days. Table II summarizes the flare information.
We created and stored 845 corresponding data samples in our database, shown in Figure 1 and accessible at https://nature.njit.edu/spacesoft/Flare-Predict/, where each data sample contains values of the 13 physical parameters or features listed in Table I. The two digits following a class label (B, C, M, X) are ignored in performing flare prediction. The time point of a data sample is the beginning time (00:00:01 early morning) of the start date of a flare and the label of the data sample is the class which the flare belongs to. These labeled data samples are used to train the DeepSun system.
|Flare Class||Number of Flares||Number of ARs|
Iii Machine Learning Algorithms
DeepSun employs three machine learning algorithms for flare prediction: random forests (RF) 
, multilayer perceptrons (MLP)[8, 9] and extreme learning machines (ELM) [10, 11]
. RF is a tree-based algorithm comprised of multiple binary classification and regression trees (CART) while both MLP and ELM are feed-forward artificial neural networks. All the three algorithms are well suited for multi-class classification. In addition, we develop an ensemble (ENS) algorithm, which works by taking the majority vote of RF, MLP and ELM. If there is no majority vote for a test data sample, ENS outputs “no verdict” for the test data sample.
Iv Performance Evaluation
We conducted a series of experiments to evaluate the performance of the machine learning algorithms presented in Section III using the database described in Section II. To avoid bias and to keep the data as balanced as possible, we created 100 csv (comma separated values) datasets of which each dataset included all B, M, and X classes and randomly selected 142 data samples of class C. We used 10-fold cross validation in which for each data set, we randomly formed 10-fold partitions using the KFold function provided by the scikit-learn library in Python . Each machine learning algorithm was trained by nine of the 10-folds, and the 10th fold was used for testing. To overcome errors associated with cross validation, we repeated the procedure 100 times for each of the 100 datasets that resulted in 10000 iterations to produce the final result.
We converted the multiple-class classification problem at hand into four binary classification problems for the four classes B, C, M, X respectively. For example, consider the binary classification problem for class B. Here, we say a data sample is positive if it is in class B, or negative if it is not in class B, i.e., it is in class C, M or X. We define TP (true positive), FP (false positive), TN (true negative), FN (false negative) as follows. TP is a data sample where an algorithm predicts the data sample to be positive and the data sample is indeed positive. FP is a data sample where the algorithm predicts the data sample to be positive while the data sample is actually negative. TN is a data sample where the algorithm predicts the data sample to be negative and the data sample is indeed negative. FN is a data sample where the algorithm predicts the data sample to be negative while the data sample is actually positive. We also use TP (FP, TN, FN respectively) to represent the number of true positives (false positives, true negatives, false negatives respectively). Because we are tackling an imbalanced classification problem (see Table II), we adopt two performance metrics, balanced accuracy (BACC) and true skill statistics (TSS), where BACC is defined as follows:
and TSS is defined as follows:
BACC and TSS are well suited for imbalanced classification of solar eruptions [13, 2, 1]. We obtain BACC and TSS for each binary classification problem. There are four binary classification problems. We then calculate the average of the BACC and TSS values obtained from the four classification problems, and use the average as the result for the multi-class classification problem.
We implemented the machine learning algorithms in Python leveraging the scikit-learn packages . Each algorithm has different optimization parameters to be tuned based on the training and test datasets. We used random forests (RF) composed of 500 to 1000 trees and set the number of features to six to find the best node split. For multilayer perceptrons (MLP) and extreme learning machines (ELM), we set the number of hidden layers to 200. These parameter values were chosen to maximize TSS values.
compares the BACC and TSS values of the machine learning algorithms at hand for each binary classification problem and for the overall multi-class classification problem where the highest performance metric values are highlighted in boldface. It can be seen from the table that the proposed ENS algorithm is better than the existing algorithms RF, MLP and ELM. However, all the four algorithms perform poorly in predicting X-class flares. This happens probably because the X class has much fewer flares than the other classes. Overall, there were approximately less than 2% data samples receiving “no verdict.”
|Class B||Class C||Class M||Class X||Average|
V The DeepSun Framework
V-a System Design
V-B System Implementation
When a user visits DeepSun’s home page, the user sees three options. Option 1 allows the user to select the pretrained models provided by DeepSun. Option 2 allows the user to upload his/her own training data to create his/her own machine learning models for solar flare prediction. Option 3 allows the user to perform solar flare prediction using RESTful services. Fig. 3 shows DeepSun’s home page, which can be accessed at https://nature.njit.edu/spacesoft/DeepSun/.
V-B1 Pretrained Models in DeepSun
The pretrained models are ready-to-use models that were created using the database described in Section II. With the pretrained models, a user has multiple options to load test data samples containing the 13 physical parameters or features listed in Table I: (1) Manually enter the data samples with values of the 13 physical parameters one by one in the provided text boxes. (2) Load sample data provided by the DeepSun engine. (3) Load the user’s own data in a file, in which each line contains the values of the 13 physical parameters. The user may invoke the services to predict all the loaded, or entered, test data at once or make predictions one by one. Fig. 4 shows the webpage of pretrained models on which four predictions were made using the ENS algorithm.
V-B2 Custom Models in DeepSun
DeepSun allows the user to load his/her data to train and create his/her custom model to predict solar flares. The training data are saved in a file meeting DeepSun’s format requirement. When the user creates a custom model, a model identifier (id) is assigned to the current session. If the created model is idle for 24 hours, it will be deleted. Once the model is ready, the user goes to the DeepSun’s graphical user interface with the assigned model id to perform flare predictions as done with the pretrained models. The model id is used to distinguish between the custom models and pretrained models. Fig. 5 shows the webpage of custom models with example training data displayed.
V-B3 RESTful API for DeepSun
The RESTful API is designed to help the programming user perform solar flare predictions using the pretrained or custom models. The API supports the POST request to predict solar flare occurrence or create a custom model, and the GET request to get a random data sample from our training database. The interface supports JSON formatted strings for requests’ body and their results. The interface also supports two different debug levels; they are (i) INFO which is the default debug mode and (ii) DEBUG to return additional data with the result.
The return result from the POST request is a JSON object including the predicted solar flare occurrence and its class. Each test data sample is associated with a JSON object that includes two attributes. One attribute is “fcnumber” which is the numerical representation for the solar flare class where we use “1” (“2”, “3”, “4” respectively) to represent class B (C, M, X respectively). The other attribute is ”fcname” which is the solar flare class name.
Vi Related Work
There are two groups of work that are closely related to ours. The first group is concerned with solar flare forecasting. Many studies in this group used parameters derived from the line-of-sight (LOS) component of the photospheric magnetic field and produced probability outputs for the occurrence of a certain magnitude flare in a time period . Some researchers  used sunspot classification and Poisson statistics to provide probabilities for an active region (AR) to produce flares with different magnitudes within 24 hours. Song et al. 
used three LOS magnetic parameters together with the ordinal logistic regression (OLR) method to predict the probabilities of a one-day flare. Bloomfieldet al.  suggested that the prediction probabilities should be converted into a binary (i.e., yes-or-no) forecast before they can be translated as flare-imminent or flare-quiet. Following this suggestion, Yuan et al. 
employed support vector machines (SVMs) to obtain a clear true or false flare prediction for different flare classes.
On the other hand, the full vector data provide more information about the photospheric magnetic field structure compared to the LOS field. This type of information may provide better flare prediction performance, but due to the limitation imposed by ground-based vector magnetic field observations, the work on flare forecast is limited. For example, Leka and Barnes  used a small sample of vector magnetograms from the Mees Solar Observatory and applied a discriminant analysis to differentiate between flare-producing and flare-quiet ARs within few hours. The authors later extended their work and used a larger number of samples with a 24-hour prediction window on producing probabilistic forecasts .
Since May 2010, the Helioseismic and Magnetic Imager (HMI) onboard the Solar Dynamics Observatory (SDO)  has been producing high quality photospheric vector magnetograms with high-cadence and full-disk coverage data. Using these data, Bobra and Couvidat  calculated a number of magnetic parameters for each AR. They selected 13 from all the available parameters and achieved good prediction performance using an SVM method for flares greater than M1.0 class. Nishizuka et al.  applied a number of machine learning algorithms to HMI data and produced prediction models for
M and X-class flares with reasonably high performance. More recently, we employed a long short-term memory network for flare prediction.
The second group of related work is concerned with services computing. Benmerar et al.  developed a brain diffusion MRI (magnetic resonance imaging) application to overcome the SaaS (software-as-a-service) limitations caused by intensive computation. The application provides APIs that tackle browser paradigms to reduce the parallel computation rendered in the client side of the browser.
Song et al. 
presented a machine learning algorithm for IT support services to automate the problem determination and classification, and also find the root cause of a problem. The algorithm is an on-line perceptron that learns about the user’s problems from the data that were generated from logs and monitoring information across different systems. The algorithm then categorizes the problems by finding the actual root cause from what it learned from the data. The algorithm employs an incremental learning technique and is able to automatically adjust the classifier parameters.
Li et al.  described a new software documentation recommendation methodology that adopts a learn-to-rank (LTR) technique. LTR is an application of supervised and semi-supervised machine learning techniques. Their strategy combines the social context from a questions-and-answers online system and the content of official software documentation to build the LTR model to provide accurate and relevant software documentation recommendations. Their experimental results showed that this approach outperforms traditional code search engines including the Google search engine.
Our DeepSun system differs from the above works in two ways. First, DeepSun provides services dedicated to solar flare prediction, which has not been addressed by the existing services computing systems. Second, in the solar flare forecasting area, DeepSun is the first MLaaS system, to our knowledge, that allows scientists to perform multi-class flare prediction through the Internet.
Vii Conclusions and Future Work
We present a machine-learning-as-a-service framework (DeepSun) for solar flare prediction. This framework provides two interfaces: a web server where the user enters the information through a graphical interface and a programmable interface that can be used by any RESTful client. DeepSun employs three existing machine learning algorithms, namely random forests (RF), multilayer perceptrons (MLP), extreme learning machines (ELM), and an ensemble algorithm (ENS) that combines the three machine learning algorithms. Our experimental results demonstrated the good performance of the ensemble algorithm and its superiority over the three existing machine learning algorithms.
In the current work we focus on data samples composed of SHARP physical parameters. We collect 845 data samples belonging to four flare classes: B, C, M, and X across 472 active regions. In addition, the Helioseismic Magnetic Imager (HMI) aboard the Solar Dynamics Observatory (SDO) produces continuous full-disk observations (solar images). In future work we plan to incorporate these HMI images into our DeepSun framework and extend our previously developed deep learning techniques[27, 28, 29] to directly process the images. We also plan to combine our recently developed deep learning algorithms using the SHARP parameters  with the image-based techniques and machine learning algorithms described in this paper for more accurate solar flare prediction.
-  C. Liu, N. Deng, J. T. L. Wang, and H. Wang, “Predicting solar flares using SDO/HMI vector magnetic data products and the random forest algorithm,” The Astrophysical Journal, vol. 843, p. 104, 2017.
H. Liu, C. Liu, J. T. L. Wang, and H. Wang, “Predicting coronal mass ejections using SDO/HMI vector magnetic data products and recurrent neural networks,”The Astrophysical Journal, vol. 890, p. 12, 2020.
-  I. Daglis, D. Baker, J. Kappenman, M. Panasyuk, and E. Daly, “Effects of space weather on technology infrastructure,” Space Weather, vol. 2, p. S02004, 2004.
-  E. Priest and T. Forbes, “The magnetic nature of solar flares,” The Astronomy and Astrophysics Review, vol. 10, pp. 313–377, 2002.
-  G. Barnes, K. D. Leka, C. J. Schrijver, T. Colak, R. Qahwaji, O. W. Ashamari, Y. Yuan, J. Zhang, R. T. J. McAteer, D. S. Bloomfield, P. A. Higgins, P. T. Gallagher, D. A. Falconer, M. K. Georgoulis, M. S. Wheatland, C. Balch, T. Dunn, and E. L. Wagner, “A comparison of flare forecasting methods. I. Results from the “all-clear” workshop,” The Astrophysical Journal, vol. 829, p. 89, 2016.
-  L. Breiman, J. H. Friedman, and R. A. Olshen, Classification and Regression Trees. CA: Wadsworth International Group, 1984.
-  M. G. Bobra, X. Sun, J. T. Hoeksema, M. Turmon, Y. Liu, K. Hayashi, G. Barnes, and K. D. Leka, “The Helioseismic and Magnetic Imager (HMI) vector magnetic field pipeline: SHARPs - Space-weather HMI Active Region Patches,” Solar Physics, vol. 289, pp. 3549–3578, 2014.
-  F. Rosenblatt, The Perceptron: A Theory of Statistical Separability in Cognitive Systems. Cornell Aeronautical Laboratory, Inc., Rep. No. VG-1196-G-1. U.S., 1958.
-  P. J. Braspenning, F. Thuijsman, and A. J. M. M. Weijters, Eds., Artificial Neural Networks: An Introduction to ANN Theory and Practice, Lecture Notes in Computer Science, vol. 931. Springer, 1995.
-  G. Huang and L. Chen, “Convex incremental extreme learning machine,” Neurocomputing, vol. 70, no. 16-18, pp. 3056–3062, 2007.
-  G.-B. Huang and L. Chen, “Enhanced random search based incremental extreme learning machine,” Neurocomputing, vol. 71, no. 16-18, pp. 3460–3468, 2008.
-  F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
-  H. Liu, C. Liu, J. T. L. Wang, and H. Wang, “Predicting solar flares using a long short-term memory network,” The Astrophysical Journal, vol. 877, p. 121, 2019.
-  P. T. Gallagher, Y.-J. Moon, and H. Wang, “Active-region monitoring and flare forecasting - I. Data processing and first results,” Solar Physics, vol. 209, pp. 171–183, 2002.
-  H. Song, C. Tan, J. Jing, H. Wang, V. Yurchyshyn, and V. Abramenko, “Statistical assessment of photospheric magnetic features in imminent solar flare predictions,” Solar Physics, vol. 254, pp. 101–125, 2009.
-  D. S. Bloomfield, P. A. Higgins, R. T. J. McAteer, and P. T. Gallagher, “Toward reliable benchmarking of solar flare forecasting methods,” The Astrophysical Journal, vol. 747, p. L41, 2012.
-  Y. Yuan, F. Shih, J. Jing, and H. Wang, “Automated flare forecasting using a statistical learning technique,” Research in Astronomy and Astrophysics, vol. 10, pp. 785–796, 2010.
-  K. D. Leka and G. Barnes, “Photospheric magnetic field properties of flaring versus flare-quiet active regions. II. Discriminant analysis,” The Astrophysical Journal, vol. 595, pp. 1296–1306, 2003.
-  G. Barnes, K. D. Leka, E. A. Schumer, and D. J. Della-Rose, “Probabilistic forecasting of solar flares from vector magnetogram data,” Space Weather, vol. 5, p. S09002, 2007.
-  M. G. Bobra and S. Couvidat, “Solar flare prediction using SDO/HMI vector magnetic field data with a machine-learning algorithm,” The Astrophysical Journal, vol. 798, p. 135, 2015.
-  N. Nishizuka, K. Sugiura, Y. Kubo, M. Den, S. Watari, and M. Ishii, “Solar flare prediction model with three machine-learning algorithms using ultraviolet brightening and vector magnetograms,” The Astrophysical Journal, vol. 835, p. 156, 2017.
-  T. Z. Benmerar, T. Megherbi, M. Kachouane, R. Deriche, and F. O. Boumghar, “Browser-level parallelism and interactive rendering APIs for scalable computation-intensive SaaS: Application to brain diffusion MRI,” IEEE Transactions on Services Computing, 2018.
-  Y. Song, A. Sailer, and H. Shaikh, “Hierarchical online problem classification for IT support services,” IEEE Transactions on Services Computing, vol. 5, no. 3, pp. 345–357, 2012.
-  J. Li, Z. Xing, and A. Kabir, “Leveraging official content and social context to recommend software documentation,” IEEE Transactions on Services Computing, 2018.
H. Liu, Y. Xu, J. Wang, J. Jing, C. Liu, J. T. L. Wang, and H. Wang, “Inferring vector magnetic fields from Stokes profiles of GST/NIRIS using a convolutional neural network,”The Astrophysical Journal, vol. 894, p. 70, 2020.
-  Z. Hu, T. Turki, N. Phan, and J. T. L. Wang, “A 3D atrous convolutional long short-term memory network for background subtraction,” IEEE Access, vol. 6, pp. 43450–43459, 2018.
Z. Hu, T. Turki, and J. T. L. Wang, “Generative adversarial networks for stochastic video prediction with action control,”IEEE Access, vol. 8, pp. 63336–63348, 2020.