PSML: A Multi-scale Time-series Dataset for Machine Learning in Decarbonized Energy Grids

10/12/2021
by   Xiangtian Zheng, et al.
0

The electric grid is a key enabling infrastructure for the ambitious transition towards carbon neutrality as we grapple with climate change. With deepening penetration of renewable energy resources and electrified transportation, the reliable and secure operation of the electric grid becomes increasingly challenging. In this paper, we present PSML, a first-of-its-kind open-access multi-scale time-series dataset, to aid in the development of data-driven machine learning (ML) based approaches towards reliable operation of future electric grids. The dataset is generated through a novel transmission + distribution (T+D) co-simulation designed to capture the increasingly important interactions and uncertainties of the grid dynamics, containing electric load, renewable generation, weather, voltage and current measurements at multiple spatio-temporal scales. Using PSML, we provide state-of-the-art ML baselines on three challenging use cases of critical importance to achieve: (i) early detection, accurate classification and localization of dynamic disturbance events; (ii) robust hierarchical forecasting of load and renewable energy with the presence of uncertainties and extreme events; and (iii) realistic synthetic generation of physical-law-constrained measurement time series. We envision that this dataset will enable advances for ML in dynamic systems, while simultaneously allowing ML researchers to contribute towards carbon-neutral electricity and mobility.

READ FULL TEXT VIEW PDF

Authors

page 1

page 7

page 10

page 11

page 12

page 15

page 18

page 22

11/29/2021

Harnessing expressive capacity of Machine Learning modeling to represent complex coupling of Earth's auroral space weather regimes

We develop multiple Deep Learning (DL) models that advance the state-of-...
03/15/2019

Multi-Stage Fault Warning for Large Electric Grids Using Anomaly Detection and Machine Learning

In the monitoring of a complex electric grid, it is of paramount importa...
01/19/2021

Critical Risk Indicators (CRIs) for the electric power grid: A survey and discussion of interconnected effects

The electric power grid is a critical societal resource connecting multi...
10/05/2020

Data-driven Operation of the Resilient Electric Grid: A Case of COVID-19

Electrical energy is a vital part of modern life, and expectations for g...
05/30/2022

A hybrid-model approach for reducing the performance gap in building energy forecasting

The performance gap between predicted and actual energy consumption in t...
06/25/2019

Future of Computing is Boring (and that is exciting!) or How to get to Computing Nirvana in 20 years or less

We see a trend where computing becomes a metered utility similar to how ...
08/09/2021

EVGen: Adversarial Networks for Learning Electric Vehicle Charging Loads and Hidden Representations

The nexus between transportation, the power grid, and consumer behavior ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The electric grid is currently one of the largest sectors of carbon emissions, and is thus expected to play a key role in tackling climate change. The electricity sector around the world is undergoing a major transition towards carbon neutrality with deepening penetration of renewable energy resources and vehicle electrification. The variability of renewable energy sources along with growing electricity demand and system vulnerability under extreme weather events pose pressing technological challenges during this transition [67]. Conventional physics-based modeling, optimization and control tools are becoming inadequate in these evolving systems due to the high degree of uncertainty and variability in power generation, consumption, and environmental factors such as climate change.

In this landscape, there is enormous scope for data-driven artificial intelligence (AI) and machine learning (ML)-based methods

[48] for tasks ranging from forecasting of renewables and load [56, 72, 61], planning [71, 43, 37], to real-time monitoring [66, 29, 30], control [16, 68] and protection [64]. Conversely, power systems are highly nonlinear dynamical systems with interesting physical phenomena at various time scales; indeed, we believe that the breadth of problems available in this domain can serve to advance the development of new algorithms, tools, and techniques in ML.

In order to foster advances that are mutually beneficial to both the ML and power system communities, it is necessary to develop well documented and calibrated open-source datasets and use cases that are relevant to real-world power engineering problems, while simultaneously being accessible and usable by ML researchers without any background in power or energy systems. There have been attempts at developing ML benchmarks for various power system tasks such as renewable

[59, 14] and load forecasting [73, 20, 2]

, and fault and anomaly detection

[75, 11, 26, 41]. Other researchers also have attempted to accelerate algorithm development by providing online simulation platforms for specific tasks, such as the L2RPN competition [49] and the oscillation source location contest [31].

However, the development of open-source datasets from the power engineering domain that can be utilized by the broader ML community are still at a nascent stage, with several gaps in existing sources. Firstly, most benchmarks for ML in power systems employ datasets that are either scattered across multiple independent system operators, as in the case of load and renewable data, or not publicly available, as in the case of disturbance or fault data. Secondly, the identification of relevant problems, the generation of the corresponding datasets, and the implementation of ML-based algorithms for scheduling, monitoring, and control, all require deep knowledge of the power engineering domain as well as many diverse power system simulation tools. This lack of coherent comprehensive datasets along with well-defined tasks is the key barrier for ML communities to contribute to power systems problems. Finally, there is a lack of consistent domain-relevant assessment metrics against which different ML algorithms can be compared.

In this paper, we bridge this gap by creating an open-source power system dataset along with associated use cases and benchmarks that are relevant to the power system research community. In particular, we focus on three areas of importance to the power engineering community, namely forecasting, monitoring, and simulating. Specifically, we consider the following use cases: (i) early detection, accurate classification and localization of dynamic disturbance events; (ii) robust hierarchical forecasting of load and renewable energy with the presence of uncertainties and extreme events; and (iii) realistic synthetic generation of physical-law-constrained measurement time series.

The dataset contains minute-level load, weather and renewable time series data of 3 years from 66 areas across the U.S., minute-level synchrophasor measurements of 1 year in 6 scenarios, and millisecond-level synchrophasor measurements in more than 1000 disturbance cases via co-simulation on a 23-bus transmission and two 13-bus distribution systems. This dataset is created by a novel joint transmission and distribution (T+D) grid simulation platform, capturing various phenomena of interest at both the bulk transmission-level and distribution level at various time scales ranging from microseconds to hours.

We note that such a joint simulation that simultaneously captures transmission and distribution system phenomena considering futuristic scenarios like very large-scale integration of electric vehicles and renewables is in and of itself novel from the power engineering perspective. In jointly simulating the bulk transmission grid and distribution grids, we are able to capture mutual interactions that cannot be observed by simulating independently. We extract, process, and condense the data from this simulation platform into time-series that can be utilized directly, independent of the simulation platform, and without any particular domain knowledge. We then benchmark the performance of both traditional algorithms from the power engineering domain and common ML-based algorithms for each of the forecasting and monitoring tasks of interest.

In summary, we develop one of the first comprehensive open-source datasets with associated use cases and benchmarks from the power systems domain that can be leveraged by ML researchers interested in advancing the state-of-the-art in time series forecasting, classification, and generation, while contributing towards future zero-carbon energy systems. The full dataset and benchmark codes can be downloaded at the Github repository 111https://github.com/tamu-engineering-research/Open-source-power-dataset.

Related work:
L2RPN [49]

provides an online interactive power grid simulation platform for reinforcement learning model training, which aims to aid in power system operations such as generation dispatch and topology change. Oscillation source location (OSL) contest

[31] provides simulated power grid oscillation data to further develop and improve the currently available OSL tools. While these datasets and platforms are largely limited to specific problems of interest, we provide a comprehensive dataset covering the entire spectrum of spatio-temporal power system dynamics, ranging from transmission to distribution-level spatial scales and millisecond to minute level time scales. We also clearly lay out use cases of interest to the power system community and the associated ML benchmarks for each of these use cases. Further, we incorporate scenarios of high renewable energy penetration and battery storage, which constitute critical enabling features distinguishing the dynamics of future carbon neutral grids from today’s power grid.

2 Challenges in Future Carbon Neutral Power Grids

Power systems are multi-scale dynamic systems from both the temporal and spatial perspectives. From the temporal perspective, power system dynamics encompass a wide range of time frames from second to a couple of months [34]. Power grid operation can thus be categorized based on different time scales, including real-time operation, day-ahead scheduling and long-term planning, each of which is typically framed and addressed using specific physical models of the grid at that time scale. From the spatial perspective, a power system consists of multiple layers of subsystems, including transmission (bulk high-voltage), subtransmission (mid-voltage), and distribution (close-to-home low-voltage) systems. Typically, it is common practice to model, simulate and analyze the subsystems independently to reduce analytical complexity. However, as the energy portfolio transitions towards large-scale integration of renewables, electric vehicles, and battery storage to mitigate climate change, both electricity generation and consumption in the future grid will become more stochastic and volatile. As a result, the mutual impact of the interaction between transmission and distribution grids can no longer be ignored [28], and critical challenges of reliability and physical security arise due to the new multi-scale, tightly coupled dynamics [67]. While traditional model-based methods may not be adequate to address these emerging challenges, there is enormous scope for the application of AI/ML-based tools in addressing problems ranging from forecasting, optimization, monitoring, and control in these future grids. Therefore, we selected three challenging tasks along with expectations for ML solutions in this paper: (i) hierarchical forecasting of load and renewable energy that is robust to uncertainties and extreme events to achieve supply-demand balance, (ii) early detection, accurate classification and localization of dynamic disturbance events to aid in the situational awareness of system operators, and (iii) realistic synthetic generation of physical-law-constrained measurement time series to mitigate the lack of real-world measurement data due to restrictive regulations.

3 Open-source Integrated T+D Power System Dataset Creation

This section briefly introduces the creation of our multi-scale time-series dataset. We first illustrate how renewable and load time series in PSML are collected. Then we describe how to generate high-fidelity voltage and current time series by co-simulating benchmark transmission and distribution systems with rich renewables. The time series generated by the co-simulation can capture the increasingly important nonlinearities and uncertainties of the grid dynamics at multiple spatio-temporal scales. More details on the data structure are documented in the Github repository.

3.1 Load and Renewable Time Series Data Generation

We collect real-world load time series and synthesize active power time series of renewable generation along with real-world weather data. For collecting the load power time series, we aggregate hourly real-world load data of representative 66 load zones ranging from 2018 to 2020, obtained from major power markets in the U.S. that regulate about 70% of U.S. electricity sales [50]. To incorporate renewable power and weather time series in PSML, we collect real weather data of 5-minute resolution of each load zone from 2018 to 2020 from National Solar Radiation Database (NSRDB) [42]

. The selected weather station of each load zone locates around major cities within range. We calculate the renewable generation power based on the collected weather data of each load zone. The active power output of residential solar photovoltaic (PV) is estimated by the System Advisor Model (SAM)

[7], based on the solar radiation-related data. The active power output of wind turbines is estimated by the location-dependent wind turbine power curves [57]

based on the collected wind-related data. Finally, we aggregate time-stamped load, renewable and weather data of 66 load zones by interpolation in the

PSML dataset. For the convenience of subsequent simulation, the load data are further normalized by their 3-year average value, while the renewable data are normalized by the nominal power values of the physical models used in the co-simulation. We illustrate daily load and renewable power profiles in Fig. 3, which shows seasonal disparity, strong variation of renewable energy, and a significant load reduction during an unprecedented event, namely the COVID-19 pandemic.

Figure 1:

Illustration of daily load, solar and wind power profiles with 1-minute resolution sampled in Houston, capturing seasonal disparity, strong variation of renewables, and unprecedented load drop during pandemic. The solid lines represent the average and shaded areas represent standard deviation.

Figure 2: Conceptual block diagram of the data flow of the transmission + distribution co-simulation platform used to create PSML. While the simulation is a closely integrated process that combines all types of input data, results with different time-scales are generated at different simulation stages.
Figure 3: Illustration of voltage magnitude time series of millisecond resolution in cases of different types of disturbances, where the black solid profiles are critical features captured by the novel T+D co-simulation, some of which are missed by the red dashed lines obtained conventional transmission system simulation alone. See the Supplementary file for more details.

3.2 Voltage and Current Measurement Time Series Data Generation

We perform joint steady-state and transient simulations between a transmission system with multiple distribution systems to reflect the increasingly significant mutual impacts between transmission and distribution systems. The time series generated by the co-simulation are capable of capturing details of quick dynamics and varying system conditions.

Minute-level Voltage, Current and Power Time Series. In the steady state simulation, circuit theory-derived algebraic equations (power flow equations) of the network are solved to obtain the voltage of nodes and the current of branches. The solution of the algebraic equations is determined by loads, renewable generation and thermal generation. The power flow equations of the transmission system and the distribution systems are respectively solved by PSS/E and OpenDSS. The correlation between the transmission and distribution systems is established by repetitively exchanging solved voltage and power between transmission and distribution (T+D) systems until the iteration process converges. Such a procedure is illustrated in Fig. 2 and see more details in the Supplementary file.

Millisecond-level Voltage, Current and Power Time Series. The dynamics of the grid can be described by differential algebraic equations (DAEs). In the simulation, the equilibrium point of the dynamics over a time period is provided by the steady state simulation. The initial conditions of the dynamic system are determined by random disturbances in the transmission system. The synthetic grid measurements are essentially the solution to the DAEs. Fig. 2 encapsulates the simulation mechanism, and Fig. 3 visualizes some typical events obtained from the co-simulation, compared with the profiles by traditional simulation. The simulation model developed for generating synthetic time series in PSML possesses the following features to obtain high-fidelity data: i) The dynamic model of both the transmission and distribution systems are benchmark systems which are extensively used in power system research; ii) we model the impact of deep penetration of renewables by incorporating detailed models of renewable generation and representative load/weather patterns in the U.S.; and iii) the interaction between the transmission and the distribution systems in the fast time scale is modeled in the simulation, which is not captured in existing publicly available synthetic datasets, e.g., [31].

Remark: The voltage and current measurement time series are simulation data. The reason that we include simulation data in the PSML dataset consists of three aspects. First, real operational data of the power grid are typically confidential and most of them are forbidden to be publicly shared, due to policies such as critical energy/electric infrastructure information (CEII). Second, some high-impact events that are challenging for analysis, such as forced oscillations under resonance conditions, are rarely observed in real-world operational data. Such a small amount of the challenging events is insufficient for the training, testing, and validation of ML algorithms. Third, while real measurement data can reflect the impact of a small amount of renewables in today’s power systems, it cannot capture the dynamics of the future grid with deep renewable penetration.

4 Use Cases and ML Benchmarks

In this section, we deliberately select three types of grid-domain use cases for ML approaches. The use cases are i) event detection, classification and localization, ii) forecasting of renewable generation and load; and iii) synthetic synchrophasor data generation. The reason that these use cases are selected is that they essentially can be formulated as classical ML problems which have been extensively studied during the past half-century. As a result, methods developed in the ML communities have great potential to provide solutions to these power grid use cases. Compared with conventional approaches heavily relying on grid physical models and network topology (line connectivity), such as the line outage detection algorithm [55], and the energy approach to forced oscillation localization [40], one attractive advantage of the ML-based approaches is that they do not require availability of information on grid physical model and topology. In addition, the sparsity, size, and scale of these time-series measurements provide a unique playground for the advancement of new ML methods. In what follows, we introduce the goal of each use case and present the benchmark of the performance of popular learning methods in terms of solving these three types of power system problems. We refer readers to our Github repository 222https://github.com/tamu-engineering-research/Open-source-power-dataset for more details on data structure and instructions on use cases and refer to the Supplementary file for more experimental results and implementation details.

4.1 Event Detection, Classification and Localization

Renewable energy resources, such as wind/solar farms, are not as dispatchable as conventional fossil fuel generators due to their stochastic nature. As a result, those renewables introduce uncertain disturbances which may compromise the safe operation of the grid. Therefore, it is imperative for Independent System Operators (ISOs) to accurately recognize disturbances and perform corrective measures timely so as to ensure the safety of the grid. The health of the power grid is monitored by sensors such as synchrophasors/phasor measurement units (PMUs). These sensors stream time-stamped measurements to ISOs. Based on these streaming measurements, ISOs may have the following two questions: i) When is an event happening; ii) What type of event is happening? and iii) Where is the source that caused the event? Answering these questions is critical to maintaining reliable operation of a power grid integrated with rich renewable energy resources.

Task Description. The streaming measurements can be denoted by , where is the number of time stamps by now and is the number of measurements. Event detection aims to answer the first ISO question by recognizing the oscillation occurrence once it takes place, hence a model is learned to be able to identify the oscillation occurrence given sequence , i.e., . Suppose the event takes place at time : when , the model is expected to be quiet without any alarms ( predicted); when , the model should alarm as soon as possible ( predicted). Event classification answers the second ISO question based on streaming sensor measurements. Given the measurement , the objective of this task is to learn a model

that can classify the underlying event type

, i.e., . In PSML, is a subset of disturbances where branch fault, branch tripping, bus fault, bus tripping, generator tripping, forced oscillation. Event localization focuses on locating events (for branch fault, branch tripping, bus fault, bus tripping, generator tripping) or the root cause of events (for forced oscillations) by observing measurements. We are aimed at learning a model that can map measurement to the bus(es) nearest to the events detected or the root cause of the events, i.e., , where is a subset of buses in the entire system. It is worth noting that compared with the size of the whole grid, the sensor coverage might be sparse in practice, rendering the tasks of event detection, classification and localization more challenging.

Metrics. With class imbalance in consideration, we adopt balanced accuracy to obtain performance of different classification methods for event classification and localization: balanced_acc = (sensitivity + specificity) / 2. Since both too early (false positive where no event happens yet) or too late (false negative where damage/cost accumulates along time without alarm or action) event detection are undesired, we leverage the macro-averaged mean absolute error, which considers the divergence between actual and predicted labels for ordinal regression on imbalanced datasets [4].

Discussion. As listed in Table 1

, we evaluate methods from four main categories: i) PMU score from the power community, ii) standard classification baselines based on 1-nearest neighbor (1-NN), iii) convolutional neural networks and iv) other popular deep learning methods. For both event classification and detection, we observe in general that approaches composed of convolutional neural networks perform much better than 1-NN based standard baselines and other deep learning approaches. However, to localize the event across the grid, the majority of deep learning approaches fail to achieve competitive performance as 1-NN approaches: only

MC-DCNN reaches above balanced accuracy besides 1-NN Euclidean, 1-NN DTW-i and 1-NN DTW-d.

In analyzing PMU measurements from the transmission system, we attribute the success of convolutional neural networks to their explicit spatial correlation modeling, where the voltage and current evolve along time according to both the external oscillation events and the inherent network connectivity. Based on the above observations, we expect more accurate classifications can be obtained by proposing more powerful deep learning methods from but not limited to the following directions: i) graph neural networks with both spatial and temporal dependencies: event localization is a great challenge when only temporal dependencies are modeled in deep learning approaches. Given bus locations and their connectivity, graph neural networks may be promising in modeling spatial dependencies and locating the actual event bus. ii) incorporating contrastive learning into representation learning: by comparing two instances ( and ) rather than learning the mapping from to

, 1-NN based approaches outperform deep learning approaches in event localization. Recently, representation learning based on comparing three instances (that is, one anchor, positive, and negative sample in triplet loss) or two instances (e.g., two similar samples by transformation or same annotated label in Siamese approaches) shows effectiveness in capturing underlying patterns and further benefits downstream tasks in domains like computer vision 

[54], reinforcement learning [24], etc. We hope that these interesting directions will motivate the ML community in addressing the challenging problems of event detection, classification and localization for dynamical systems that exhibit tight multi-scale spatio-temporal coupling.

[capbesideposition=left,top]table[]

Table 1: Classification performance on simulation measurements for event detection, classification and localization. We present values for experiments with random seeds.

width=0.8 Categories Methods Classification Localization Detection (Balanced Acc) (Balanced Acc) (Macro MAE) Power Domain PMU score [47] 0.266 Traditional Machine Learning Methods 1-NN Euclidean [5, 12] 0.537 0.402 36.465 1-NN DTW-i [5, 12] 0.610 0.463 53.928 1-NN DTW-d [5, 12] 0.598 0.474 53.709 MiniRocket [13] 0.690 0.022 0.208 0.226 53.908 3.358 Convolutional Neural Networks Vanilla CNN 0.564 0.058 0.168 0.053 40.458 12.686 InceptionTime [18] 0.715 0.040 0.243 0.047 43.743 10.605 MLSTM-FCN [32] 0.742 0.029 0.285 0.023 31.873 5.400 ResNet [62] 0.725 0.049 0.232 0.044 38.578 9.569 MC-DCNN [78] 0.726 0.019 0.437 0.030 38.107 5.675 TapNet [76] 0.653 0.018 0.397 0.065 58.251 1.974 Other Deep Learning Methods Fully-connected Neural Network 0.583 0.042 0.245 0.035 54.131 9.964 Vanilla RNN 0.504 0.045 0.224 0.037 57.184 4.285 LSTM [27] 0.544 0.049 0.248 0.043 56.434 2.851 GRU [10] 0.653 0.029 0.332 0.062 55.550 2.090 Vanilla Transformer [58] 0.612 0.041 0.340 0.090 46.824 0.866

[capbesideposition=left,top]table[]

Table 2: Performance on t+60 (1 hour ahead) load point forecast and interval prediction. Performance on t+1440 load, t+5 and t+30 wind/solar is listed in Table 1 to 5 in Supplementary.

width= Categories Methods Point Forecast 95% Pred. Interval RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.041 0.039 0.032 0.029 0.029 0.024 0.032 0.032 0.027 0.203 0.175 0.190 ARIMA 0.031 0.027 0.024 0.022 0.020 0.017 0.023 0.022 0.019 0.113 0.095 0.083 ETS 0.029 0.026 0.022 0.021 0.019 0.017 0.022 0.021 0.018 0.107 0.088 0.076 Traditional Machine Learning Methods SVR 0.057 0.058 0.044 0.046 0.048 0.037 0.053 0.057 0.043 0.181 0.182 0.185 RF 0.033 0.032 0.028 0.026 0.026 0.022 0.029 0.028 0.024 0.099 0.093 0.095 GBDT 0.033 0.032 0.024 0.026 0.026 0.020 0.029 0.029 0.023 0.095 0.087 0.091 LR 0.026 0.027 0.030 0.020 0.022 0.024 0.021 0.024 0.026 1.141 0.871 0.100 Multilayer Perceptron ELM 0.229 0.145 0.184 0.196 0.119 0.146 0.233 0.140 0.170 0.531 0.427 0.573 FNN 0.088 0.091 0.114 0.073 0.076 0.085 0.083 0.089 0.100 0.240 0.248 0.257 N-BEATS [46] 0.090 0.091 0.084 0.067 0.072 0.068 0.072 0.082 0.077 0.209 0.207 0.236 Convolutional Neural Networks Vanilla CNN 0.103 0.065 0.089 0.085 0.049 0.058 0.099 0.053 0.067 0.263 0.243 0.243 WaveNet [45] 0.159 0.109 0.123 0.132 0.087 0.103 0.156 0.093 0.120 0.438 0.454 0.659 TCN [36] 1.336 0.966 1.133 1.330 0.960 1.130 1.478 1.100 1.277 2.777 1.996 2.359 Recurrent Neural Networks Vanilla RNN 0.154 0.094 0.084 0.122 0.075 0.068 0.137 0.088 0.080 0.330 0.273 0.230 LSTM [27] 0.070 0.136 0.167 0.055 0.114 0.101 0.061 0.136 0.120 0.244 0.286 0.302 GRU [10] 0.087 0.132 0.092 0.071 0.108 0.074 0.081 0.127 0.086 0.216 0.286 0.201 LSTNet [35] 0.065 0.056 0.0826 0.050 0.046 0.067 0.055 0.053 0.075 0.188 0.328 0.227 DeepAR [51] 0.068 0.110 0.093 0.055 0.086 0.078 0.059 0.101 0.090 1.752 1.515 1.531 Transformer -based Transformer [58] 0.108 0.121 0.132 0.086 0.095 0.101 0.098 0.114 0.119 0.265 0.280 0.298 Informer [79] 0.129 0.110 0.073 0.102 0.086 0.059 0.118 0.101 0.067 0.252 0.215 0.214 - Neural ODE [9] 0.117 0.265 0.151 0.098 0.249 0.128 0.102 0.270 0.137 0.737 0.709 0.801

4.2 Load and Renewable Energy Forecasting

The ultimate goal of the power grid is to balance generation and load. Today, this is mostly achieved by load forecasting and generation scheduling based on the forecast before real-time operation. In real-time operation, the relatively small mismatch between scheduled generation and actual load is compensated by dispatchable generation units that can respond quickly, i.e., the spinning reserve. The spinning reserve may rely on fossil fuel and incur high operational costs. However, with increasing renewable integration, this operational paradigm is not feasible without accurate renewable and load forecasting. Renewable generation, e.g., wind/solar farms, are considered as negative load and cannot be actively dispatched, due to their stochastic and volatile nature. A poor forecast of renewable generation therefore leads to a large amount of expensive, fossil fuel-based spinning reserve being committed. Compounding the challenge, loads will become less predictable in the future grid, due stochastic loads like electric vehicles, and small-probability yet high-impact events, e.g., COVID-19 pandemic (Fig.

1) and the Texas winter storm in 2021. Therefore, accurate forecasting of renewable generation and load is critical to support reliable operation of the future grid.

Task Description. We focus on the following two important subtasks: i) Point Forecast (PF): given a time sequence in the past time steps, the current time and the forecasting horizon , we have the past targets , the past observations , as well as the past, current and future known variables (e.g., date, holiday). We aim to predict the targets time steps ahead: . ii) Prediction Interval (PI): Uncertainty quantification can provide more reference information for decision-making and has received growing research and industrial interests these years [33, 39]. For high-quality uncertainty quantification, we would also like to obtain the prediction interval, , to cover the ground truth with at least the expected tolerance probability, i.e., in our load and renewable energy forecasting task.

Metrics. For point forecast, we adopt three commonly leveraged metrics in load and renewable energy forecasting literature [1]: root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE). Following practice in the M4 competition [39], the performance of generated point intervals is evaluated using the mean interval score (MIS) [22]: , where is the number of instances for each prediction horizon, is the indicator function with value when the inequality holds and otherwise, and for prediction intervals generation.

Discussion. We list the performance of different forecasting methods on 1-hour ahead load forecasting in years from 2018 to 2020 in Table 2 (the performance on more forecasting tasks is displayed in Supplementary file). For short-term load forecasting (Table 2), exponential smoothing outperforms the other baselines both in point forecast and prediction interval, while deep learning approaches fail to achieve competitive performance on par with time series models and traditional machine learning methods. Similar observations can be discovered in both short- and long-term forecasting of wind and solar.

As visualized in Fig. 5 (in the Supplementary file), we can observe strong periodicity in both the observational features (such as wind speed, relative humidity, and temperature) as well as target features (such as solar power, wind power, and load) from the year 2018 to year 2020. However, deep learning approaches fail to capture such significant trends for accurate future forecasting due to their limited memorizing capabilities. For extremely long time series, it’s difficult to efficiently extract and leverage useful past time steps without tedious feature engineering [60]. Taking our forecasting task as an example: simply enlarging the scope of considered historical data to cover information from the previous -th, -th, -th time step in our minute-level data for the reference of the same time in the last day, same weekday in the previous week, and same day in the previous month, is both time-consuming during processing crowded useless information and will deteriorate forecasting performance in the end. Accordingly, we suggest the following directions for further exploration: i) memory network to remember and utilize past history efficiently: approaches such as memory networks [63] could be potential solutions to identify key information from long histories for real-time forecasting. ii) cross-learning: using information from multiple series to predict individual ones has shown promising results in top approaches of past Kaggle competitions [8]. Taking into account load and renewable energy time series from other locations, such as nearby locations in the same time zone or with similar social and economic patterns, could potentially enhance forecasting accuracy. Thus, this task can motivate the development of novel deep learning based forecasting algorithms that take into account long-term memory and exploit spatio-temporal patterns in the data.

4.3 Synthetic Time-series Generation

A major hurdle in applying deep learning models to power system problems is usually the lack of sufficient and high-quality datasets for training, as it is well-known that more eventful data usually lead to better classification performance [21, 44, 77]. The accessibility of real-world power grid PMU measurement data is limited due to the regulation CEII [19] for national security and sensitivity concerns. While researchers recently have contributed to the creation of large-scale synthetic simulation models [6] for analysis [70, 65], there are always gaps between simulation models and real-world systems and the unique values of real-world PMU time series data cannot be exploited for research purposes. It is therefore critical to investigating methods for synthesizing power system datasets that follow the same properties of the real system data while complying with physical laws for the network and its underlying dynamic behaviors.

Task Description. This task involves multi-channel time series generation, for which the training data are disturbance-induced dynamic voltage, current and power measurements across power grids. The expected outputs are dynamic voltage, current and power measurements that preserve certain dynamic patterns and physical laws. The key challenges that distinguish this task from normal image generation are: i) multi-channel time series are governed by unknown algebraic and differential equations derived from physical laws; and ii) dynamic time series incorporate discrete disturbance events. Our evaluations are carried out over simulated voltage, current and power data from PSML.

[capbesideposition=left,top]table[]

Table 3: Run-time and fidelity metrics on millisecond-level eventful PMU datasets. Auto- and cross-correlation are calculated as the sum of the absolute difference between the correlation coefficients computed from real and generated samples.

width=0.65 Method Autocorrelation Cross- Discriminative Hours correlation Score NaiveGAN [23] 663.13 7175.78 0.492 0.022 5 RCGAN [17] 667.89 7607.29 0.499 0.002 27 COT-GAN [69] 112.42 2532.69 0.435 0.016 6 TimeGAN [74] 72.56 1361.45 0.481 0.008 52 DoppelGANger [38] 86.76 3994.45 0.447 0.004 22

[.4]

Figure 4: Autocorrelation on PMU datasets for all generative methods.

[.4]

Figure 5: Distribution of generated faults for DoppelGANger.
Figure 6: PCA coverage evaluation on millisecond-level eventful PMU datasets.

Metrics. The task is to synthesize multiple realistic-looking PMU streams that respect the physical constraints from real PMU streams. To assess the quality of generated data, we measure: (i) Fidelity - samples should be indistinguishable from the real data. We train a post-hoc time-series classification model (by optimizing a 2-layer LSTM) to distinguish between sequences from the original and generated datasets and report the error. (ii) Diversity - samples should be distributed to cover the real data. We apply PCA analyses on both the original and synthetic datasets (flattening the temporal dimension) and visualize how closely the distributions are in 2D space.

Discussion. We observe that current SOTA time-series generation methods cannot properly capture the necessary characteristics from PMU data to generate realistic time-series. This is reflected in the fidelity metrics in Table 3 where a post-hoc 2-layer LSTM can easily separate real vs. generated samples. Time-GAN [74] and COT-GAN [69] are current SOTA methods published in NeurIPS 19 and 20 respectively and they both achieve relatively low auto-correlation and cross-correlation error compared to real data (Fig. 5). However, a quick qualitative inspection of the generated samples in Supplementary A shows that the generated samples are quite different from real data. In addition, we apply PCA analyses on both the original and synthetic datasets (flattening the temporal dimension) and visualize how closely the distributions are in 2D space (Fig. 6). We observe that overall, the methods also fail to cover the underlying data distribution. DoppelGANger [38] is the only method that can model metadata, i.e., fault type in this case, and can generate sensible results. However, it still struggles to learn the distribution of the metadata (Fig. 7). One of the main challenges from this time-series dataset is its size (dimensionality and sequence lengths). Compared to the datasets from their original papers, fast-sampled PMU data from PSML nearly double or triple the number of observations at 960 observations with 91 channels for measurements including voltage magnitude, voltage phase angle, current magnitude, current phase angle, real power, reactive power, and frequency. Current generation approaches leveraging recurrent networks do a poor job of modelling the long-term temporal correlations seen in the data. For long time series, RNNs take many passes to generate the entire samples, which causes them to forget long temporal correlation. Besides, one particular challenge in the power grid data is that each dimension of the time series cannot be handled separately, since the whole system is governed by the Kirchhoff’s voltage and current laws at each snapshot. This provides an interesting direction for future generation work to address not only the scalability problem but also the constrained generation problem.

5 Conclusion & Broader Impact

We presented a multi-scale time-series dataset that integrates transmission and distribution networks for realistic power grids encompassing futuristic scenarios of large-scale renewable energy integration and dynamic loads such as electric vehicles. To bridge the gap between the ML and power system communities, we identified three categories of use cases that solve important practical problems of relevance to the power and energy community and benchmarked state-of-the-art ML algorithms for these cases. While real power data are notoriously difficult to release due to security regulations over grid reliability, simulated data over a synthetic grid allow researchers to study grid properties without fearing of network infrastructure being reverse engineered. We envision that this dataset will facilitate mutual interaction between the ML and power systems communities, simultaneously advancing the state-of-the-art in ML-based algorithms for forecasting, event classification, and constrained synthetic data generation, while contributing towards reliable operation of a carbon-neutral electric grid. Future work will incorporate electricity market features to the dataset and enrich the use cases by adding market-relevant use cases, such as forecasting local marginal prices.

References

  • [1] Tanveer Ahmad, Hongcai Zhang, and Biao Yan. A review on renewable energy and electricity requirement forecasting models for smart grid and buildings. Sustainable Cities and Society, 55:102052, 2020.
  • [2] Abdulaziz Almalaq and George Edwards. A review of deep learning methods applied on load forecasting. In 2017 16th IEEE international conference on machine learning and applications (ICMLA), pages 511–516. IEEE, 2017.
  • [3] Martin Arjovsky, Soumith Chintala, and Léon Bottou.

    Wasserstein generative adversarial networks.

    In International conference on machine learning, pages 214–223. PMLR, 2017.
  • [4] Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. Evaluation measures for ordinal regression. In 2009 Ninth international conference on intelligent systems design and applications, pages 283–287. IEEE, 2009.
  • [5] Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075, 2018.
  • [6] Adam B Birchfield, Ti Xu, Kathleen M Gegner, Komal S Shetye, and Thomas J Overbye. Grid structural characteristics as validation criteria for synthetic networks. IEEE Transactions on power systems, 32(4):3258–3265, 2016.
  • [7] Nate Blair, Aron P Dobos, Janine Freeman, Ty Neises, Michael Wagner, Tom Ferguson, Paul Gilman, and Steven Janzou. System advisor model, sam 2014.1. 14: General description. Technical report, National Renewable Energy Lab.(NREL), Golden, CO (United States), 2014.
  • [8] Casper Solheim Bojer and Jens Peder Meldgaard. Kaggle forecasting competitions: An overlooked learning opportunity. International Journal of Forecasting, 37(2):587–603, 2021.
  • [9] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. arXiv preprint arXiv:1806.07366, 2018.
  • [10] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  • [11] Mingjian Cui, Jianhui Wang, and Meng Yue. Machine learning-based anomaly detection for load forecasting under cyberattacks. IEEE Transactions on Smart Grid, 10(5):5724–5734, 2019.
  • [12] Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The ucr time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
  • [13] Angus Dempster, Daniel F Schmidt, and Geoffrey I Webb. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 248–257, 2021.
  • [14] Alberto Dolara, Francesco Grimaccia, Sonia Leva, Marco Mussetta, and Emanuele Ogliari. Comparison of training approaches for photovoltaic forecasts by means of machine learning. Applied Sciences, 8(2):228, 2018.
  • [15] R. C. Dugan and D. Montenegro. The open distribution system simulator™(opendss), reference guide. https://sourceforge.net/p/electricdss/code/HEAD/tree/trunk/Distrib/Doc/OpenDSSManual.pdf, 2018.
  • [16] Rayan El Helou, Dileep Kalathil, and Le Xie. Fully decentralized reinforcement learning-based control of photovoltaics in distribution grids for joint provision of real and reactive power. IEEE Open Access Journal of Power and Energy, 8:175–185, 2021.
  • [17] Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633, 2017.
  • [18] Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and François Petitjean. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery, 34(6):1936–1962, 2020.
  • [19] Federal Energy Regulatory Commission. Critical Energy/Electric Infrastructure Information. Available: https://www.ferc.gov/ceii [Online], 2021.
  • [20] Cong Feng and Jie Zhang. Assessment of aggregation strategies for machine-learning based short-term load forecasting. Electric Power Systems Research, 184:106304, 2020.
  • [21] Zwe-Lee Gaing. Wavelet-based neural network for power disturbance recognition and classification. IEEE transactions on power delivery, 19(4):1560–1568, 2004.
  • [22] Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359–378, 2007.
  • [23] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
  • [24] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733, 2020.
  • [25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In

    Proceedings of the IEEE conference on computer vision and pattern recognition

    , pages 770–778, 2016.
  • [26] Raymond C Borges Hink, Justin M Beaver, Mark A Buckner, Tommy Morris, Uttam Adhikari, and Shengyi Pan. Machine learning for power system disturbance and cyber-attack discrimination. In 2014 7th International symposium on resilient control systems (ISRCS), pages 1–8. IEEE, 2014.
  • [27] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  • [28] Qiuhua Huang and Vijay Vittal. Integrated transmission and distribution system power flow and dynamic simulation using mixed three-sequence/three-phase modeling. IEEE Transactions on Power Systems, 32(5):3704–3714, 2016.
  • [29] Tong Huang, Nikolaos M. Freris, P. R. Kumar, and Le Xie. A synchrophasor data-driven method for forced oscillation localization under resonance conditions. IEEE Transactions on Power Systems, 35(5):3927–3939, 2020.
  • [30] Tong Huang, Sicun Gao, Xun Long, and Le Xie. A neural lyapunov approach to transient stability assessment in interconnected microgrids. In Proceedings of the 54th Hawaii International Conference on System Sciences, page 3330, 2021.
  • [31] IEEE PES, NASPI. 2021 IEEE-NASPI oscillation source location contest. Available: http://web.eecs.utk.edu/~kaisun/Oscillation/2021Contest/ [Online], 2021.
  • [32] Fazle Karim, Somshubra Majumdar, Houshang Darabi, and Samuel Harford. Multivariate lstm-fcns for time series classification. Neural Networks, 116:237–245, 2019.
  • [33] Abbas Khosravi, Saeid Nahavandi, Doug Creighton, and Amir F Atiya. Comprehensive review of neural network-based prediction intervals and new advances. IEEE Transactions on neural networks, 22(9):1341–1356, 2011.
  • [34] Prabha Kundur. Power system stability. Power system stability and control, pages 7–1, 2007.
  • [35] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 95–104, 2018.
  • [36] Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks for action segmentation and detection. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 156–165, 2017.
  • [37] Mingxuan Li, Wei Wei, Yue Chen, Ming-Feng Ge, and Joao P. S. Catalao. Learning the optimal strategy of power system operation with varying renewable generations. IEEE Transactions on Sustainable Energy, pages 1–1, 2021.
  • [38] Zinan Lin, Alankar Jain, Chen Wang, Giulia Fanti, and Vyas Sekar. Using gans for sharing networked time series data: Challenges, initial promise, and open questions. In Proceedings of the ACM Internet Measurement Conference, pages 464–483, 2020.
  • [39] Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54–74, 2020.
  • [40] Slava Maslennikov, Bin Wang, and Eugene Litvinov. Dissipating energy flow method for locating the source of sustained oscillations. International Journal of Electrical Power & Energy Systems, 88:55–62, 2017.
  • [41] Mostafa Mohammadpourfard, Yang Weng, and Mohsen Tajdinian. Benchmark of machine learning algorithms on capturing future distribution network anomalies. IET Generation, Transmission & Distribution, 13(8):1441–1455, 2019.
  • [42] National Renewable Energy Laboratory. NSRDB: National Solar Radiation Database. Available: https://nsrdb.nrel.gov/ [Online], 2021.
  • [43] Chao Ning and Fengqi You. Deep learning based distributionally robust joint chance constrained economic dispatch under wind power uncertainty. IEEE Transactions on Power Systems, pages 1–1, 2021.
  • [44] Hatice Okumus and Fatih M Nuroglu. Power system event classification based on machine learning. In 2018 3rd International Conference on Computer Science and Engineering (UBMK), pages 402–405. IEEE, 2018.
  • [45] Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, 2016.
  • [46] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437, 2019.
  • [47] Shikhar Pandey, Anurag K Srivastava, and Brett G Amidan. A real time event detection, classification and localization using synchrophasor data. IEEE Transactions on Power Systems, 35(6):4421–4431, 2020.
  • [48] David Rolnick, Priya L Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, et al. Tackling climate change with machine learning. arXiv preprint arXiv:1906.05433, 2019.
  • [49] RTE, EPRI, and CHA LEARN. L2RPN challenge. Available: https://l2rpn.chalearn.org/competitions [Online], 2021.
  • [50] Guangchun Ruan, Dongqi Wu, Xiangtian Zheng, Haiwang Zhong, Chongqing Kang, Munther A Dahleh, S Sivaranjani, and Le Xie. A cross-domain approach to analyzing the short-run impact of covid-19 on the us electricity sector. Joule, 4(11):2322–2337, 2020.
  • [51] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020.
  • [52] K. P. Schneider et.al. Analytic considerations and design basis for the ieee distribution test feeders. IEEE Transactions on Power Systems, 33(3):3181–3188, 2018.
  • [53] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi.

    Inception-v4, inception-resnet and the impact of residual connections on learning.

    In Thirty-first AAAI conference on artificial intelligence, 2017.
  • [54] Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701–1708, 2014.
  • [55] Joseph Euzebe Tate and Thomas J. Overbye. Line outage detection using phasor angle measurements. IEEE Transactions on Power Systems, 23(4):1644–1652, 2008.
  • [56] Jean-François Toubeau, Jérémie Bottieau, François Vallée, and Zacharie De Grève. Deep learning-based multivariate probabilistic forecasting for short-term scheduling in power markets. IEEE Transactions on Power Systems, 34(2):1203–1215, 2019.
  • [57] U.S. Energy Information Administration. Form EIA-860 with previous formdata. Available: https://www.eia.gov/electricity/data/eia860/ [Online], 2021.
  • [58] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.
  • [59] Cyril Voyant, Gilles Notton, Soteris Kalogirou, Marie-Laure Nivet, Christophe Paoli, Fabrice Motte, and Alexis Fouilloy. Machine learning methods for solar radiation forecasting: A review. Renewable Energy, 105:569–582, 2017.
  • [60] Bin Wang, Jie Lu, Zheng Yan, Huaishao Luo, Tianrui Li, Yu Zheng, and Guangquan Zhang. Deep uncertainty quantification: A machine learning approach for weather forecasting. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2087–2095, 2019.
  • [61] Xinan Wang, Yishen Wang, Di Shi, Jianhui Wang, and Zhiwei Wang. Two-stage wecc composite load modeling: A double deep q-learning networks approach. IEEE Transactions on Smart Grid, 11(5):4331–4344, 2020.
  • [62] Zhiguang Wang, Weizhong Yan, and Tim Oates. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International joint conference on neural networks (IJCNN), pages 1578–1585. IEEE, 2017.
  • [63] Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv preprint arXiv:1410.3916, 2014.
  • [64] Dongqi Wu, Xiangtian Zheng, Dileep Kalathil, and Le Xie. Nested reinforcement learning based control for protective relays in power distribution systems. In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 1925–1930, 2019.
  • [65] Dongqi Wu, Xiangtian Zheng, Yixing Xu, Daniel Olsen, Bainan Xia, Chanan Singh, and Le Xie. An open-source extendable model and corrective measure assessment of the 2021 texas power outage. Advances in Applied Energy, 4:100056, 2021.
  • [66] Le Xie, Yang Chen, and P. R. Kumar. Dimensionality reduction of synchrophasor data for early event detection: Linearized analysis. IEEE Transactions on Power Systems, 29(6):2784–2794, 2014.
  • [67] Le Xie, Chanan Singh, Sanjoy K Mitter, Munther A Dahleh, and Shmuel S Oren. Toward carbon-neutral electricity and mobility: Is the grid infrastructure ready? Joule, 2021.
  • [68] Hanchen Xu, Alejandro D. Domínguez-García, and Peter W. Sauer. Optimal tap setting of voltage regulation transformers using batch reinforcement learning. IEEE Transactions on Power Systems, 35(3):1990–2001, 2020.
  • [69] Tianlin Xu, Li K Wenliang, Michael Munn, and Beatrice Acciaio. Cot-gan: Generating sequential data via causal optimal transport. arXiv preprint arXiv:2006.08571, 2020.
  • [70] Yixing Xu, Nathan Myhrvold, Dhileep Sivam, Kaspar Mueller, Daniel J Olsen, Bainan Xia, Daniel Livengood, Victoria Hunt, Benjamin Rouillé d’Orfeuil, Daniel Muldrew, et al. US test system with high spatial and temporal resolution for renewable integration studies. In 2020 IEEE Power & Energy Society General Meeting (PESGM), pages 1–5. IEEE, 2020.
  • [71] Ziming Yan and Yan Xu. Real-time optimal power flow: A <italic>lagrangian</italic> based deep reinforcement learning approach. IEEE Transactions on Power Systems, 35(4):3270–3273, 2020.
  • [72] Chen Yang, Anupam A. Thatte, and Le Xie. Multitime-scale data-driven spatio-temporal forecast of photovoltaic generation. IEEE Transactions on Sustainable Energy, 6(1):104–112, 2015.
  • [73] Baran Yildiz, Jose I Bilbao, and Alistair B Sproul. A review and analysis of regression and machine learning models on commercial building electricity load forecasting. Renewable and Sustainable Energy Reviews, 73:1104–1122, 2017.
  • [74] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. 2019.
  • [75] Ameema Zainab, Shady S Refaat, Dabeeruddin Syed, Ali Ghrayeb, and Haitham Abu-Rub. Faulted line identification and localization in power system using machine learning techniques. In 2019 IEEE International Conference on Big Data (Big Data), pages 2975–2981. IEEE, 2019.
  • [76] Xuchao Zhang, Yifeng Gao, Jessica Lin, and Chang-Tien Lu. Tapnet: Multivariate time series classification with attentional prototypical network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6845–6852, 2020.
  • [77] Xiangtian Zheng, Bin Wang, Dileep Kalathil, and Le Xie. Generative adversarial networks-based synthetic pmu data creation for improved event classification. IEEE Open Access Journal of Power and Energy, 8:68–76, 2021.
  • [78] Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. Time series classification using multi-channels deep convolutional neural networks. In International conference on web-age information management, pages 298–310. Springer, 2014.
  • [79] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI, 2021.

Supplementary

Appendix A The Psml Dataset

We publish the PSML dataset, benchmark models and data documentation at the Github Repository https://github.com/tamu-engineering-research/Open-source-power-dataset. We bear all responsibility in case of violation of rights, etc., and confirmation of the data license.

Dataset documentation.

Please refer to the Github repository for the dataset documentation and intended uses.

Terms of use, privacy and license.

The PSML dataset is published under CC BY-NC 4.0 license, meaning everyone can use it only for non-commercial research purpose.

Data maintenance.

The Github repository provides data download links for users. We will keep maintaining and updating the data for a long time and check the data accessibility.

Benchmark and code.

We provide benchmark models in the Github repository. The reproduction code will be released upon acceptance.

Appendix B Details of Simulation Model

Fig. 7 visualizes the co-simulated PSS/E 23-bus transmission system and the IEEE 13-bus distribution system. The model of the bulk transmission grid is a modified version of the PSS/E 23 bus test system, which is a well-configured high-voltage transmission grid that has been commonly used as a benchmark for transmission system studies. The original system has 6 thermal generators and 7 load buses. Two load buses connect to detailed distribution systems and the rest of the load buses are connected to lumped load. To better comply with the ratings and size of the benchmark distribution systems, all load capacities are reduced by roughly 40% of their original values. The dynamic model of a thermal generator is replaced by a wind turbine model. The IEEE 13 node feeder [52] is used to represent distribution systems connected to the load buses in the PSS/E 23 bus transmission system model. External PV models written in Python scripts are attached to the load buses in distribution system. Siemens PSS/E, a commercial power grid simulation software, is used to simulate the transmission system model and OpenDSS [15], an open-source three-phase unbalanced distribution system simulator, is used to simulate two distribution system models. A Python control process is used to facilitate workflow coordination and data exchange between PSS/E, OpenDSS and input data files.

Figure 7: Diagram of joint simulation between transmission and distribution (T+D) systems, with real load profiles and simulated renewable profiles as input. Transmission systems represent bulk high-voltage grids while distribution represents close-to-home low-voltage grids.

We create random disturbance to induce transient PMU measurements under these fault events. More specifically, Fig. 8 visualizes different types of disturbances in transient co-simulation that respectively correspond to transient profiles illustrated in Fig. 3 in the main paper.

Figure 8: Visualization of different types of disturbances in the PSS/E 23-bus transmission system that induce the transient millisecond-level voltage, current and power measurements.

Appendix C Baseline Performance

c.1 Use Case 1: Disturbance Event Classification and Localization

Baseline algorithms

  1. Power domain: Event localization is implemented by calculating the event signature of each PMU, where the event signature is estimated by several statistical parameters including Shannon entropy, standard deviation, range, mean difference and crest factor as introduced in the literature [47]. The PMU with the most dominant event signature indicates the location of the event.

  2. Traditional machine learning methods: with generally good performance across different time series datasets, 1-nearest neighbor (1-NN) related approaches have been widely employed as standard baselines in the their corresponding benchmarks, e.g., UCR 333https://www.cs.ucr.edu/~eamonn/time_series_data_2018/ [12] and UEA 444http://www.timeseriesclassification.com/index.php [5]. We consider three measures for sample distance computation: Euclidean and dynamic time warping with each feature dimension treated independently (DTW-i) or dependently (DTW-d). We also adopt MiniRocket, where kernel transformations are firstly applied to time series followed by simple linear classifiers for time series classification [13].

  3. Convolutional Neural Networks: benefiting from the deep convolutional neural networks (CNNs) and residual connections, vanilla and different variants of CNNs are consider to perform classification tasks: 1) InceptionTime [18]: an ensemble of deep CNNs inspired by Inception-v4 [53] architecture; 2) MLSTM-FCN [32]: concatenation of LSTM of CNN for feature representation learning with an additional squeeze-and-excitation block for further performance improvement; 3) ResNet [62]: adaptation of residual network from images [25] to multivariate time series; 4) MC-DCNN [78]: multi-channels deep CNNs where different temporal patterns are transformed firstly and then learned through separate convolutional layers; and 5) TapNet [76]: an attentional prototype network was incorporated into the convolutional layers to learn latent features for time series classification.

  4. Other deep learning methods: we also analyze the performance of general deep neural networks (i.e., fully-connected neural network) and specific time series deep models (i.e., RNN and its two variants [27, 10], transformer [58]) is evaluated.

c.2 Use Case 2: Load and Renewable Forecasting

Baseline algorithms

  1. Time-series models: besides the naive method takes the current value directly as the prediction, we also consider autoregressive integrated moving average (ARIMA) and exponential smoothing (ETS).

  2. Traditional machine learning methods: we select the top four widely used machine learning methods in load and renewable energy forecasting literature [1]

    : support vector regression (SVR), random forest (RF), gradient boosted decision trees (GBDT), and linear regression (LR).

  3. Multilayer perceptron: besides fully-connected neural newotks (FNN) and extreme learning machines (ELM), we also list performance of N-BEATS, a deep stacked neural architecture based on backward and forward residual links, which outperformed the winner of M4 competition [46].

  4. Convolutional Neural Networks: we study performance of vanilla CNN, WaveNet [45] composed of dilated causal convolutions for audio generation, Temporal convolutional neural networks (TCN [36]) with additional residual blocks.

  5. Recurrent Neural Networks

    : both basic recurrent neural networks (vanilla RNN, LSTM 

    [27] and GRU [10]) and advanced variants are studied: LSTNet [35] with patterns extracted from convolutional layers and fed to recurrent neural networks, DeepAR [51] with output from recurrent neural networks as likelihood parameters for probabilistic forecasting.

  6. Transformer-based: we list performance from vanilla transformer [58] and its variant, informer [79], with self-attention distilling and generative style decoder for long sequence forecasting.

  7. Neural ODE: motivated by the Euler discretization of continuous transformations in residual networks and recurrent neural network decoders, Neural ODEs parameterize the derivative of hidden state using a neural network and compute the network output with a differential equation solver [9]

c.3 Use Case 3: Synthetic PMU Generation

Baseline algorithms

  1. NaiveWGAN: we show the performance of a naive GAN architecture (MLP generator and discriminator) with the Wasserstein loss [3].

  2. RCGAN: a conditional recurrent GAN architecture [17] is tested that leverage recurrent generator and discriminator and conditioned on auxiliary information.

  3. COT-GAN: we test a recurrent GAN trained with a Causal Optimal Transport (COT) loss suitable for learning time dependent data distributions [69].

  4. TimeGAN

    : we list the performance of a recurrent GAN architecture that combines unsupervised GAN learning with a supervised teacher-forcing component in the loss function

    [74].

  5. DoppelGANger: we test a state-of-the-art GAN architecture [38] that leverages two generators and discriminators to first generate auxiliary metadata before generating the time-series.

Figure 9: Real and generated samples from different GAN models for synthetic time-series generation. (1/2)
Figure 10: Real and generated samples from different GAN models for synthetic time-series generation. White boxes in RCGAN and NaiveGAN shown above are a result of these methods failing to capture the proper range for voltage measurements. (2/2)

width= Categories Methods Point Forecast (PFs) 95% Prediction Intervals (PIs) RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.081 0.068 0.055 0.055 0.047 0.035 0.061 0.052 0.040 0.357 0.297 0.297 ARIMA 0.999 0.942 0.782 0.730 0.713 0.592 0.790 0.792 0.654 4.016 3.320 2.580 ETS 0.081 0.068 0.055 0.055 0.047 0.035 0.061 0.052 0.040 0.143 0.184 0.287 Traditional Machine Learning Methods SVR 0.082 0.075 0.059 0.066 0.060 0.048 0.075 0.071 0.056 0.326 0.291 0.303 RF 0.061 0.057 1.541 0.047 0.043 1.424 0.053 0.050 1.575 0.213 0.209 2.998 GBDT 0.062 0.058 0.056 0.048 0.044 0.041 0.054 0.052 0.047 0.201 0.201 0.235 LR 0.081 0.143 0.240 0.056 0.117 0.141 0.062 0.133 0.158 3.532 4.339 3.458 Multilayer Perceptron ELM 0.204 0.135 0.171 0.172 0.112 0.133 0.204 0.130 0.154 0.492 0.421 0.558 FNN 0.099 0.100 0.106 0.082 0.083 0.087 0.094 0.099 0.101 0.391 0.298 0.338 N-BEATS 0.085 0.079 0.093 0.066 0.068 0.075 0.072 0.080 0.083 0.247 0.314 0.318 Convolutional Neural Networks Vanilla CNN 0.096 0.082 0.091 0.079 0.065 0.069 0.090 0.072 0.079 0.362 0.329 0.319 WaveNet 0.285 0.155 0.246 0.215 0.122 0.215 0.232 0.136 0.251 0.913 0.676 0.638 TCN 1.334 0.964 1.134 1.328 0.958 1.131 1.472 1.094 1.280 2.772 1.992 2.362 Recurrent Neural Networks Vanilla RNN 0.129 0.108 0.076 0.106 0.090 0.064 0.124 0.107 0.073 0.356 0.341 0.311 LSTM 0.120 0.145 0.170 0.094 0.124 0.106 0.110 0.147 0.122 0.385 0.364 0.349 GRU 0.100 0.118 0.076 0.080 0.097 0.062 0.092 0.117 0.071 0.318 0.314 0.234 LSTNet 0.099 0.086 0.082 0.075 0.070 0.064 0.086 0.080 0.072 0.358 0.372 0.272 DeepAR 0.132 0.181 0.142 0.117 0.162 0.130 0.130 0.188 0.148 1.567 1.210 1.289 Transformer -based Vanilla Transformer 0.097 0.101 0.103 0.081 0.084 0.080 0.090 0.099 0.093 0.374 0.305 0.329 Informer 0.084 0.081 0.070 0.067 0.066 0.056 0.076 0.076 0.062 0.280 0.244 0.274 - Neural ODE 0.165 0.308 0.188 0.133 0.286 0.158 0.137 0.309 0.170 1.862 1.486 1.794

Table 4: Performance on t+1440 (1 day ahead) load point forecast and interval prediction.

width=1.1 Categories Methods Point Forecast (PFs) 95% Prediction Intervals (PIs) RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.00354 0.00385 0.00557 0.00124 0.00154 0.00229 0.08874 0.08023 0.08351 0.01875 0.01884 0.01874 ARIMA 0.00435 0.00494 0.00686 0.00152 0.00195 0.00275 0.11870 0.10722 0.10763 0.00336 0.00644 0.00880 ETS 0.00366 0.00385 0.00557 0.00134 0.00154 0.00229 0.10809 0.08025 0.08353 0.00297 0.00462 0.00639 Traditional Machine Learning Methods SVR 0.05794 0.06274 0.08260 0.04450 0.04668 0.05639 12.157 9.1356 9.1497 0.16125 0.16262 0.11831 RF 0.05333 0.06357 0.09083 0.01678 0.02410 0.04097 0.53469 0.47324 0.60916 0.10965 0.05773 0.08850 GBDT 0.05327 0.06425 0.08958 0.01670 0.02544 0.03959 0.51495 0.79686 0.55692 0.10686 0.05949 0.08462 LR 0.00288 0.00330 0.00486 0.00110 0.00142 0.00200 0.14442 0.13953 0.09925 0.03000 0.01876 0.00540 Multilayer Perceptron ELM 0.05794 0.05444 0.06242 0.04617 0.04250 0.04835 12.895 7.7364 6.95903 0.48166 0.43765 0.43144 FNN 0.08376 0.05129 0.11466 0.05242 0.03394 0.08657 7.67768 4.55800 8.61012 0.27371 0.24528 0.28085 N-BEATS 0.06706 0.04315 0.09049 0.04722 0.02918 0.06509 9.45264 4.13055 6.4516 0.25272 0.23815 0.26589 Convolutional Neural Networks Vanilla CNN 0.07672 0.04095 0.04655 0.05353 0.02856 0.03253 8.84433 4.11337 3.95313 0.22156 0.29990 0.24129 WaveNet 0.15164 0.14840 0.17448 0.10975 0.13928 0.16518 25.32545 30.1669 36.53756 0.55639 0.55370 0.47946 TCN 0.70497 0.73707 0.68061 0.70304 0.73460 0.67558 198.46 159.75 149.44 1.45484 1.51941 1.39240 Recurrent Neural Networks Vanilla RNN 0.09759 0.07978 0.13271 0.07022 0.05214 0.10840 11.1344 6.67483 15.198 0.37834 0.29442 0.33508 LSTM 0.05718 0.06011 0.07394 0.03727 0.04538 0.05355 5.92506 5.4723 5.68631 0.28757 0.32075 0.36095 GRU 0.06601 0.05118 0.11549 0.04092 0.03319 0.07806 5.26180 3.42500 8.68033 0.23244 0.26014 0.30979 LSTNet 0.05718 0.06011 0.07394 0.03727 0.04538 0.05355 5.92506 5.47239 5.68631 0.28757 0.32075 0.36095 DeepAR 0.87163 0.86246 0.89018 0.86105 0.85268 0.88205 243.76 191.25 196.80 0.21182 0.21845 0.25903 Transformer -based Vanilla Transformer 0.05179 0.06612 0.0881 0.02316 0.04778 0.06068 2.32431 6.6665 6.34723 0.28751 0.29035 0.28987 Informer 0.04238 0.05557 0.06345 0.02397 0.03664 0.04271 3.81838 4.20209 5.25698 0.25627 0.27453 0.26340 - Neural ODE 0.08718 0.09452 0.11768 0.06406 0.06876 0.08841 16.2678 13.4499 12.321 1.38396 1.86382 1.72871

Table 5: Performance on t+5 (5 minute ahead) wind point forecast and interval prediction.

width= Categories Methods Point Forecast (PFs) 95% Prediction Intervals (PIs) RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.0114 0.0121 0.0180 0.0041 0.0049 0.0079 0.2469 0.2150 0.2638 0.0625 0.0628 0.0632 ARIMA 0.0255 0.0282 0.0400 0.0094 0.0118 0.0169 0.7732 0.6881 0.7381 0.0219 0.0288 0.0474 ETS 0.0114 0.0121 0.0180 0.0041 0.0049 0.0079 0.2469 0.2150 0.2638 0.0087 0.0119 0.0179 Traditional Machine Learning Methods SVR 0.0579 0.0627 0.0826 0.0445 0.0466 0.0564 12.149 9.1378 9.1524 0.1898 0.1590 0.1183 RF 0.0537 0.0640 0.0914 0.0175 0.0249 0.0418 0.6607 0.5790 0.7058 0.1291 0.0623 0.0911 GBDT 0.0538 0.0641 0.0913 0.0186 0.0258 0.0416 1.1855 0.9650 0.7087 0.1346 0.0621 0.0901 LR 0.0101 0.0102 0.0196 0.0052 0.0055 0.0094 1.0195 0.7778 0.6331 0.1054 0.0613 0.0212 Multilayer Perceptron ELM 0.0617 0.0507 0.0645 0.0510 0.0357 0.0502 14.742 5.7782 7.4985 0.4790 0.4239 0.4430 FNN 0.0872 0.0525 0.1232 0.0554 0.0328 0.0907 8.4005 3.9938 9.2386 0.2934 0.2584 0.3015 N-BEATS 0.0704 0.0455 0.0992 0.0498 0.0300 0.0714 9.4754 4.3082 7.2831 0.2717 0.2542 0.2959 Convolutional Neural Networks Vanilla CNN 0.0730 0.0419 0.0475 0.0526 0.0284 0.0337 9.0781 4.0255 4.3893 0.2341 0.3103 0.2691 WaveNet 0.0826 0.1294 0.1055 0.0728 0.1244 0.0965 19.865 27.924 19.674 0.4495 0.4879 0.4775 TCN 0.7049 0.7370 0.6805 0.7030 0.7345 0.6755 198.34 159.78 149.48 1.4550 1.5199 1.3924 Recurrent Neural Networks Vanilla RNN 0.0989 0.0883 0.1395 0.0717 0.0698 0.1128 12.034 11.165 16.672 0.3801 0.3046 0.3505 LSTM 0.0599 0.0929 0.1329 0.0294 0.0575 0.0926 3.9967 4.9654 15.194 0.2877 0.3308 0.3235 GRU 0.0635 0.0548 0.1215 0.0398 0.0362 0.0859 5.5690 3.9050 10.684 0.2489 0.2742 0.3192 LSTNet 0.0617 0.0681 0.0753 0.0413 0.0534 0.0538 6.6415 6.1077 5.6686 0.2968 0.3308 0.3721 DeepAR 0.9083 0.9019 0.8633 0.8964 0.8941 0.8561 251.36 195.15 188.19 0.2753 0.4002 0.3761 Transformer -based Vanilla Transformer 0.0519 0.0709 0.0886 0.0252 0.0536 0.0625 3.2199 8.1435 7.4363 0.2982 0.3015 0.3060 Informer 0.0520 0.0592 0.0693 0.0342 0.0413 0.0498 5.3202 7.2401 6.3545 0.2614 0.3158 0.2907 - Neural ODE 0.1175 0.1310 0.1390 0.1061 0.1197 0.1224 32.254 28.554 28.679 0.9588 1.1884 1.3126

Table 6: Performance on t+30 (half hour ahead) wind point forecast and interval prediction.

width= Categories Methods Point Forecast (PFs) 95% Prediction Intervals (PIs) RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.0348 0.0485 0.0449 0.0198 0.0266 0.0236 0.7555 0.2535 0.2721 0.1341 0.1758 0.1467 ARIMA 0.0456 0.0685 0.0598 0.0168 0.0286 0.0209 0.9393 0.2563 0.2719 0.3815 0.5929 0.1239 ETS 0.0348 0.0485 0.0465 0.0198 0.0266 0.0201 0.7555 0.2535 0.2114 0.2980 0.4247 0.1130 Traditional Machine Learning Methods SVR 0.0866 0.1107 0.0965 0.0728 0.0885 0.0746 4.1220 1.9381 1.5404 0.3995 0.4427 0.2527 RF 0.1196 0.0527 0.0458 0.0716 0.0318 0.0218 0.5913 1.0497 0.9100 0.3491 0.6679 0.1072 GBDT 0.1061 0.0651 0.0448 0.0648 0.0469 0.0192 1.5019 1.3056 0.7210 0.3398 0.7193 0.1061 LR 0.0292 0.0509 0.0581 0.0147 0.0323 0.0218 0.4143 0.6488 0.2879 0.2097 0.1879 0.0948 Multilayer Perceptron ELM 0.2162 0.1835 0.1825 0.1889 0.1534 0.1559 17.545 5.9924 4.8913 0.7930 0.7530 0.6922 FNN 0.2150 0.2035 0.2051 0.1835 0.1709 0.1763 4.8498 4.2713 1.8218 0.4768 0.5213 0.5147 N-BEATS 0.2186 0.2186 0.1933 0.1868 0.1857 0.1624 4.5870 4.5044 4.5576 0.4704 0.5425 0.5066 Convolutional Neural Networks Vanilla CNN 0.2266 0.1985 0.2326 0.1858 0.1651 0.1948 19.224 4.7316 7.5374 0.4931 0.4703 0.5367 WaveNet 0.1778 0.1539 0.1876 0.1514 0.1341 0.1655 6.5645 4.4543 4.7301 0.7086 0.7606 0.7413 TCN 0.5095 0.5759 0.4934 0.4614 0.5366 0.4451 36.617 17.219 14.359 1.0664 1.1879 1.0501 Recurrent Neural Networks Vanilla RNN 0.2692 0.3052 0.2253 0.2235 0.2627 0.1741 21.438 6.6038 6.3623 0.6112 0.7072 0.7208 LSTM 0.2852 0.2845 0.2384 0.2514 0.2506 0.1907 16.974 8.5450 6.7934 0.5789 0.5992 0.5802 GRU 0.2194 0.2186 0.2144 0.1846 0.1840 0.1812 5.7645 5.2248 4.3416 0.4668 0.4854 0.4907 LSTNet 0.2178 0.2016 0.2132 0.1946 0.1747 0.1882 14.680 6.1754 5.2839 0.7106 0.7395 0.6639 DeepAR 0.7467 0.8138 0.6950 0.7124 0.7861 0.6580 49.720 22.690 19.096 0.6213 0.7264 0.6630 Transformer -based Vanilla Transformer 0.2218 0.2400 0.1994 0.1863 0.1927 0.1689 20.166 5.7722 5.2274 0.4911 0.5712 0.4785 Informer 0.2412 0.2407 0.2281 0.2063 0.1978 0.1891 3.6477 4.6121 3.1762 0.5030 0.5156 0.5037 - Neural ODE 0.2610 0.2141 0.2611 0.2232 0.1784 0.2223 10.120 4.7316 4.0280 1.4171 1.3872 1.4679

Table 7: Performance on t+5 (5 minute ahead) solar point forecast and interval prediction.

width= Categories Methods Point Forecast (PFs) 95% Prediction Intervals (PIs) RMSE MAE MAPE MIS 2018 2019 2020 2018 2019 2020 2018 2019 2020 2018 2019 2020 Time-series Models Naive 0.0942 0.1054 0.1036 0.0716 0.0779 0.0784 3.8826 0.9770 1.2037 0.3630 0.4016 0.3754 ARIMA 0.0456 0.3987 0.3546 0.0168 0.1909 0.1407 0.9393 1.7802 2.3500 0.3815 3.5134 0.7873 ETS 0.0942 0.1054 0.1043 0.0716 0.0779 0.0654 3.8826 0.9770 1.0965 0.7304 0.7433 0.2846 Traditional Machine Learning Methods SVR 0.1162 0.1366 0.1137 0.0945 0.1135 0.0910 5.4285 3.5780 3.6830 0.5692 0.4896 0.2537 RF 0.1018 0.3010 0.0999 0.0655 0.2408 0.0588 3.5810 1.4906 3.5003 0.7313 0.9354 0.1866 GBDT 0.1220 0.1566 0.0981 0.0862 0.1147 0.0553 7.4917 3.7402 3.1172 0.6544 0.9751 0.1786 LR 0.0797 0.1196 0.1090 0.0541 0.0932 0.0609 3.2401 1.3085 1.0247 0.6104 0.5436 0.2130 Multilayer Perceptron ELM 0.2117 0.2002 0.1914 0.1838 0.1685 0.1619 16.737 5.4905 5.1410 0.8028 0.8181 0.7307 FNN 0.2296 0.2056 0.2203 0.1923 0.1738 0.1847 3.4088 2.9114 2.4649 0.5139 0.4911 0.5594 N-BEATS 0.2248 0.2327 0.1921 0.1904 0.1955 0.1595 3.5577 3.3240 4.8967 0.4988 0.5614 0.5446 Convolutional Neural Networks Vanilla CNN 0.2194 0.1944 0.2290 0.1785 0.1604 0.1909 19.056 3.2488 7.3985 0.4979 0.4672 0.5417 WaveNet 0.1757 0.1833 0.2009 0.1547 0.1543 0.1762 8.7785 3.2808 4.9382 0.7168 0.8124 0.8008 TCN 0.4947 0.5650 0.4783 0.4475 0.5255 0.4310 34.105 13.554 14.276 1.0437 1.1651 1.0231 Recurrent Neural Networks Vanilla RNN 0.2696 0.3111 0.2440 0.2233 0.2624 0.1860 17.031 6.5575 8.3913 0.6466 0.7613 0.8413 LSTM 0.2823 0.3048 0.2526 0.2459 0.2646 0.1977 14.501 7.2775 8.6101 0.5825 0.6342 0.6524 GRU 0.2303 0.2138 0.2142 0.1972 0.1791 0.1784 3.8142 3.9333 4.7827 0.5098 0.4768 0.5040 LSTNet 0.2156 0.1982 0.2132 0.1917 0.1700 0.1868 13.736 4.7210 5.3596 0.7599 0.7873 0.7331 DeepAR 0.5445 0.5739 0.4769 0.4932 0.5273 0.4154 36.992 13.865 15.486 0.6512 0.4955 0.6314 Transformer -based Vanilla Transformer 0.2164 0.2303 0.2012 0.1847 0.1815 0.1692 19.945 4.1457 4.0780 0.50435 0.5435 0.5116 Informer 0.2438 0.2450 0.2266 0.2052 0.2029 0.1870 4.0555 3.2678 3.2251 0.5277 0.5419 0.5395 - Neural ODE 0.2424 0.2081 0.2449 0.2078 0.1709 0.2111 10.536 3.2220 2.8119 1.1744 1.5559 1.5466

Table 8: Performance on t+30 (half hour ahead) solar point forecast and interval prediction.

Appendix D Benchmark Implementation

d.1 Use Case 1: Event Detection, Classification and Localization

We implement InceptionTime, MC-DCNN and ResNet (with the help of sktime-dl 555https://github.com/sktime/sktime-dl.git package), and MLSTM-FCN666https://github.com/houshd/MLSTM-FCN.git

in Tensorflow 

777https://www.tensorflow.org/

. We implemented all other methods by ourselves in Pytorch 

888https://pytorch.org/ except TapNet 999https://github.com/xuczhang/tapnet.git and MiniRocket 101010https://github.com/angus924/minirocket.git

. For deep learning approaches, we use grid search to select general hyperparameters such as layer size, number of layers, normalization approach, etc.

Dataset Splitting Training time-series are extracted from the millisecond transient PMU data. We randomly select 439 time-series as training samples and the remaining 110 time-series for testing. Each sequence has a metadata associated with the event type similar to the classification use case, i.e. branch_fault, branch_trip, bus_fault, bus_trip, gen_trip. Each time-series has a sequence length of 960 observations, representing 4s in the system recorded at 240Hz. There are 91 dimensions for each time-series, including voltage, current and power measurements across the transmission system.

d.2 Use Case 2: Load and Renewable Forecasting

We implement the time-series models with statsmodels 111111https://www.statsmodels.org/stable/index.html, the traditional machine learning models with sklearn 121212https://scikit-learn.org/, N-BEATS 131313https://github.com/ElementAI/N-BEATS.git, WaveNet 141414https://github.com/vincentherrmann/pytorch-wavenet.git, TCN 151515https://github.com/locuslab/TCN.git, LSTNet 161616https://github.com/laiguokun/LSTNet.git, DeepAR 171717https://github.com/zhykoties/TimeSeries.git, Informer 181818https://github.com/zhouhaoyi/Informer2020.git and Neural ODE 191919https://github.com/rtqichen/torchdiffeq.git with codes published officially (or unofficially), and all other deep learning approaches by ourselves in Pytorch. For deep learning approaches, we use grid search to select general hyperparameters such as layer size, number of layers, normalization approach, etc.

Dataset Splitting Given load and renewable energy data recorded from 66 locations in minute-level, we split the sequence from each location according to years firstly and have three cases: 1) use data from Jan to Nov in 2018 for training and Dec in 2018 for testing; 2) use data from Jan, 2018 to Nov, 2019 for training and Dec in 2019 for testing; 3) use data from Jan, 2019 to Nov, 2020 for training and Dec in 2020 for testing. Noted that we adopt the rolling strategy during testing, so that testing data before current time step is observable for model forecasting. As shown in Fig. 11, besides target features for load, wind and solar, we also provide date-related known variables (i.e., month of day, day of the week, holiday indicator) and weather-related observations (i.e., DHI, DNI, GHI, dew point, solar zenith angle, wind speed, relative humidity and temperature).

d.3 Use Case 3: Synthetic PMU Generation

We run all methods on an 4 NVIDIA RTX 2080 Ti with 44GB of memory and all models are trained with a fixed hidden dimensionality of 256, a fixed number of two or three layers for recurrent networks, and a tuned dropout ratio {0.0, 0.5}.

Dataset Splitting: Training time-series are extracted from the millisecond transient PMU data. We set the first 400 time-series as training samples and the next 150 time-series for testing. Each sequence have a metadata associated with the event type similar to the classification use case, i.e. branch_fault, branch_trip, bus_fault, bus_trip, gen_trip. Each time-series has a sequence length of 960 observations, representing 4s in the system recorded at 240Hz. There are 91 dimensions for each time-series.

Figure 11: Visualization of feature trends in three years from one sampled location of the forecasting task.