I-a A Brief Overview of Cyber-Attacks and Countermeasures in Smart Grid
Due to the integration of advanced signal processing, communication, and control technologies, smart grid relies on a critical cyber infrastructure that is subject to adversarial cyber threats [1, 2, 3, 4]. The smart grid is regulated based on estimated system states and the main aim of attackers is to damage/mislead the state estimation mechanism and thereby to cause wrong/manipulated decisions in the energy management system of the smart grid. Some potential consequences of a successful cyber-attack are regional power blackouts, manipulated electricity market prices [5, 6], and destabilization of the power grid . Such cyber-attacks are also seen in practice. For instance, on December 23, 2015, the Ukrainian power system was attacked and the resulting power blackout affected around 200,000 people for several hours .
The Ukraine attack has demonstrated that attackers have more capabilities than predicted . Namely, (i) attackers can access and monitor the power system over long periods of time without being detected, (ii) attackers are able to perform cyber-attacks by hacking smart grid components (smart meters, control centers, etc.), manipulating/jamming the network communication channels, and accessing and manipulating the database of the control center [8, 9, 2]. Hence, cyber-attacks significantly threaten the safe and reliable operation of the power grid in practice. Effective countermeasures need to be developed considering the worst-case scenarios where the attackers are fully capable of performing a diverse range of cyber-attacks. The first step in a defense mechanism is early detection of cyber-attacks. After detecting an attack, effective mitigation schemes should then be implemented.
against the smart grid are extensively studied in the literature and several detectors are proposed. The proposed detectors are mostly outlier detectors, i.e., they classify a sample measurement as either normal or anomalous. Conventional detectors classify a measurement as anomalous if the measurement residual exceeds a certain threshold[16, 10, 17, 18, 19]
. More advanced machine learning techniques are also considered for classification of anomalous measurements[20, 21]. Moreover, in , firstly a Markov graph model for system states is learned under normal system operation and then attacks/anomalies are detected based on the consistency of new measurements compared to the learned nominal model. Further, in , based on the least squares (LS) state estimator, a multi-step procedure is presented to detect and classify cyber-attacks on meter measurements, network line parameters, and network topology, and then to make corrections for attack mitigation.
In [23, 24, 25], robust extended Kalman filters have been proposed where the main aim is to bound the effects of outliers on the state estimation mechanism. No specific attack types are considered so that using such schemes, it is not possible to distinguish a real attack from random outliers, e.g., due to heavy-tailed non-Gaussian noise processes. Moreover, such schemes have breakdown points such that if outliers, significantly far away from the nominal measurements, are observed, then the proposed filters fail to keep track of the system state.
In order to improve the time resolution and also to detect cyber-attacks more reliably, several online detectors based on the quickest detection theory are proposed. For instance, in  and , cumulative sum (CUSUM)-based schemes are considered to detect FDI attacks where the state estimation is based on the conventional LS methods. More recently, in , CUSUM-based detection schemes are proposed to detect FDI and denial of service (DoS) attacks (separately) in a dynamic setting and their advantages over the outlier detectors and the LS-based detectors are demonstrated. Further, in , a nonparametric CUSUM detector is proposed that do not assume any attack model and only evaluates the deviation of meter measurements from the baseline statistics, i.e., normal system operation. In , a window-based CUSUM detector is proposed for detection of FDI attacks where the attack parameters of interest are estimated based on the most recent sliding window of measurements.
In this paper, we propose robust mechanisms for timely detection of potentially combined and stealthily designed FDI and jamming attacks. The proposed mechanisms are tightly connected to an estimation mechanism, which makes both the detection and state estimation schemes robust against unknown and time-varying attack variables. In particular, online maximum likelihood estimates (MLEs) of the attack types, set of attacked meters, and the attack magnitudes are used in attack detection. Moreover, recovered state estimates are computed based on the online MLE estimates of the attack variables. No restrictive assumptions are made about an attacker’s strategy, i.e., an attacker can design and perform arbitrarily combined FDI and jamming attacks, targeting any subset of meters in any magnitude and can also change its attack parameters over time. Further, considering the possibility of smarter and more capable attackers, additional countermeasures are proposed against stealthily designed cyber-attacks. These make the proposed detection schemes highly robust against a significantly wide range of potential cyber-attacks targeting the smart grid.
Since the smart grid is a highly complex network, any anomaly/failure in a part of the system can quickly spread over the network and lead to new unpredicted failures. Hence, timely attack detection and mitigation is crucial. In this paper, for timely detection, we present real-time detection mechanisms. Moreover, to help for timely attack mitigation and quick system recovery, we provide online estimates of the attack types, set of attacked meters and attack magnitudes. Note that having an estimate for the attack type can be useful since different countermeasures may need to be employed against different types of attacks. Further, considering that the real power grid is a huge network consisting of many meters, an estimate of the attacked meters can be critical for a timely and effective attack mitigation, e.g., via isolating the attacked meters during the recovery procedure. Moreover, estimates of attack magnitudes are needed to recover attack-free states.
We list our main contributions as follows:
A novel low-complexity online detection and estimation algorithm is proposed against (possibly) combined FDI and jamming attacks. The proposed algorithm is robust to unknown and time-varying attack types, magnitudes, and set of attacked meters. Further, recovered state estimates and closed-form online MLE estimates of the attack variables are presented.
Stealthy attacks against CUSUM-based detectors and particularly against the proposed algorithm are introduced and analyzed.
Several countermeasures are proposed against the considered stealthy attacks.
The remainder of the paper is organized as follows. In Sec. II, the system model, attack models, state estimation mechanism, and the problem formulation are presented. In Sec. III, an online cyber-attack detection and estimation algorithm is presented. In Sec. IV, stealthy attacks against CUSUM-based detectors are introduced and analyzed. Also, countermeasures against the considered stealthy attacks are presented. In Sec. V, the proposed detection schemes are evaluated extensively via simulations. Finally, the paper is concluded in Sec. VI
. Boldface letters denote vectors and matrices, and all vectors are column vectors.
Ii System Model and Problem Formulation
Ii-a System Model
The actual power grid is regulated based on a nonlinear AC power flow model . On the other hand, the approximate linearized (around an operating point) DC power flow model is a good approximation that is widely used in the literature to describe the operation of the power grid [16, 10, 31]. Furthermore, static system model and consequently conventional static (LS) state estimation are not effective in capturing the dynamics of a power system due to time-varying load and power generation . In addition, attack detection mechanisms based on static estimators are not effective in detecting time-varying cyber-attacks and structured “stealth” FDI attacks , for which dynamic state estimator-based detectors are known to be effective [28, 32].
where is the state vector denoting the phase angles of buses (one of the buses is considered as a reference bus), is the state transition matrix, is the process noise vector, is an identity matrix, and is the transpose operator. Further, is the vector consisting of meter measurements, is the measurement vector for meter , is the measurement matrix, is the measurement noise vector, and is the measurement noise vector for meter . Note that in each time interval between and , measurements are taken at each meter, where is usually small, and the collected measurements between and are processed at time . To increase the measurement redundancy against noise and also to estimate the unknown attack parameters more reliably in case of a cyber-attack, needs to be chosen higher.
In general, the state transition and measurement matrices can also be dynamic. For instance, due to changes in network topology, i.e., on and off states of the switches and line breakers in the power grid, the measurement matrix may vary over time. In that case, instead of modeling the smart grid as a linear time-invariant system as in (1) and (2), we can model it as a linear time-varying system where we can replace and by and , respectively. The results presented in this study can be generalized to the case of linear time-varying system model as long as and are known by the system controller at each time .
Ii-B Attack Models
We assume that at an unknown time , a cyber-attack is launched to the system, where we particularly consider FDI attacks, jamming attacks, and their combination. The attack types, attack magnitudes, and the set of attacked meters can be time-varying. But, during a time interval, i.e., between and , we assume that the attack parameters stay constant. Next, we explain the attack models under consideration.
Ii-B1 FDI Attack
In case of an FDI attack, additive malicious data are injected into the measurements of a subset of meters. In practice, an FDI attack can be performed by manipulating the network communication channels or hacking meters and/or control centers in the smart grid [2, 8]. The measurement model in case of an FDI attack takes the following form:
where denotes the injected false data at time . Since the attack magnitudes are assumed to be constant between and , for meter , , where is a vector consisting of s. Note that if meter is not under an FDI attack at time , then , otherwise .
Ii-B2 Jamming Attack
In case of a jamming attack, we assume that the attacker constantly emits additive white Gaussian noise (AWGN) to the network communication channels to compromise a subset of meter measurements. We consider jamming with AWGN since (i) it is a commonly employed jamming model in the literature [33, 34]
, (ii) it is a simple attacking strategy to perform, and (iii) in an additive noise channel with Gaussian input, for a given mean and variance, among all noise distributions, the Gaussian noise maximizes the mean squared error of estimating the channel input given the channel output[35, 34]. Hence, an attacker can jam the communication channels with AWGN to maximize its damage on the state estimation mechanism.
In case of a jamming attack, the measurement model can be written as follows:
where denotes the jamming noise, , and where is the variance of the jamming noise targeting meter at time . If meter is not under a jamming attack at time , then , otherwise .
Ii-B3 Hybrid Attack
In case of a hybrid (combined) attack, FDI and jamming attacks are simultaneously launched to the system and hence the measurement model takes the following form:
For meter under both FDI and jamming attacks at time , and . Since the FDI and jamming attacks can be considered as special cases of hybrid attacks, we consider (5) as the measurement model under the attacking regime, i.e., for .
)) and the jamming noise terms are mutually independent over the meters, then the considered hybrid FDI/jamming attacks span all possible data attacks. This is due to the fact that a Gaussian random variable is defined by its mean and variance, and through the hybrid attacks, mean and variance of the density of meter measurements can be arbitrarily changed (cf. (5)). For instance, in case of a DoS attack, meter measurements are blocked and only a random or zero signal is received at the control center [9, 13, 14]. Hence, the DoS attack can be considered as a special case of the hybrid cyber-attacks, i.e., a DoS attack can either be equivalent to an FDI attack with false data being in the same magnitude of the actual signal but with an opposite sign or a jamming attack with high level noise variances such that the actual signal can be neglected compared to the noise signal 
. On the other hand, if the jamming noise is correlated over the meters or it is not normally distributed, then such an attack does not comply with the considered jamming attack model in (4) and nor with (5). For such cases, we consider a non-parametric goodness-of-fit test as a countermeasure (see Sec. IV-C2).
Ii-C Pre- and Post-Attack Measurement Densities
Let where is the measurement matrix for meter . Since the measurement matrix is determined based on the system topology, the rows of are identical, i.e., , where is a row of . Based on the considered post-attack model in (5), a measurement obtained at meter during the time interval between and , i.e., can be written as
where is the set of non-attacked meters, is the set of meters under only FDI attack, is the set of meters under only jamming attack, and is the set of meters under both FDI and jamming attacks at time . Note that , , , and are disjoint sets and .
Then, the probability density functions (pdfs) of the measurements in the pre- and post-attack regimes take respectively the following forms:
Ii-D State Estimation
Since the smart grid is modeled as a discrete-time linear dynamic system with the Gaussian noise terms (cf. (1) and (2)), the Kalman filter is the optimal linear estimator in minimizing the mean squared state estimation error . Further, since the measurement models for the pre- and post-attack periods are different (cf. (7) and (8)), two Kalman filters need to be simultaneously employed: one for assuming no attack occurs at all and one for assuming an attack occurs at an unknown time . Since the latter involves the unknown change-point and the unknown attack parameters and , estimates of these unknowns are needed to employ the corresponding Kalman filter. As we will explain later, is estimated by the detection algorithm, and are estimated via the maximum likelihood (ML) estimation.
The Kalman filter is an iterative real-time estimator composed of prediction and measurement update steps at each iteration. Let the Kalman filter estimates for the pre- and post-attack cases be denoted with and , respectively where and for the prediction and measurement update steps at time , respectively. The Kalman filter equations at time are then given as follows:
Pre-attack – Prediction:
Pre-attack – Measurement update:
Post-attack – Prediction:
Post-attack – Measurement update:
where and denote the estimates of the state covariance matrix at time , and and denote the Kalman gain matrices at time for the pre- and post-attack cases, respectively. Note that the MLE estimates of the attack parameters are used in the measurement update step of the Kalman filter for the post-attack case, where is the MLE of (cf. (26)) and is the MLE of (cf. (27)). Hence, and are, in fact, recovered state estimates in case of a cyber-attack. Note, however, that ML estimation errors may lead to errors in computing the recovered state estimates.
Ii-E Problem Formulation
Our objective is detecting cyber-attacks in a timely and reliable manner and the quickest detection theory [37, 38, 39] is well suited to this objective. In the quickest change detection problems, measurements become available sequentially over time and at each time, either a change is declared or further measurements are taken in the next time interval, where the aim is to optimally balance the detection delay and the false alarm rate. There are two main approaches in the quickest detection theory, namely Bayesian and non-Bayesian. In a Bayesian setting, the change point is considered as a random variable with a known a priori distribution whereas in a non-Bayesian setting, the change point is considered as non-random and unknown. Our problem better fits to the non-Bayesian setting since we do not assume any a priori knowledge about the change-point . Then, we consider the following objective function, proposed by Lorden :
where is the stopping time at which an attack is declared, denotes all measurements obtained up to time , and is the expectation under , that is the probability measure if the change occurs at time . Note that is called the worst-case average detection delay since it is maximized over the change point and also over all measurements obtained up to the change-point. We then consider the following minimax optimization problem:
where is called the average false alarm period, i.e., average stopping time when no change occurs at all (), and is a prespecified lower bound for .
Let the pre- and post-attack measurement pdfs given in (7) and (8) be denoted with and , respectively. Since the dynamic system state is not directly observed and the attack parameters and are completely determined by an attacker and hence unknown, both pdfs are unknown and time-varying. If the pre- and post-attack pdfs would be exactly known, then the well-known CUSUM algorithm would be the optimal solution to (14) . Nonetheless, the system state can be inferred using the Kalman filters and the MLEs of the unknown attack parameters can be computed. Then, following a generalized likelihood ratio approach [38, Sec. 5.3], [26, 28] and replacing the unknowns with their estimates, a generalized CUSUM algorithm can be used as a solution to (14).
In this paper, in addition to early attack detection, we also aim to recover the attack-free system states. Notice that in case of no attack, i.e., for , the Kalman filter for the pre-attack case (assuming no attack at all) is the optimal state estimator. However, after an attack occurs, the measurement model assumed in the pre-attack period (cf. (2)) is no longer true. Hence, the state estimates for the pre-attack case, i.e., and , deviate from the actual system state for . Recalling that an attack occurs at an unknown time and the measurements follow the post-attack measurement model (cf. (5)) for , if the attack launch time and the attack magnitudes and would be exactly known, then the system state would be perfectly recovered for . Nonetheless, as we will explain more clearly in the next section, the (generalized) CUSUM algorithm always keeps a change-point estimate in its memory and updates this estimate as the measurements become sequentially available over time [38, Sec. 2.2]. When an attack is declared at the stopping time , becomes the final change-point estimate of the (generalized) CUSUM algorithm. Furthermore, the MLEs of the attack magnitudes, i.e., and , can be computed at each time . Then, employing a Kalman filter for the post-attack case (cf. (11) and (12)) and computing the state estimates using the MLEs of the attack parameters in the measurement update step, recovered state estimates, i.e., and , can be obtained for .
Iii Online Attack Detection and Estimation
Since it is hard to distinguish noise from FDI/jamming attacks with small magnitudes, some minimum levels for the attack magnitudes need to be defined in order to control the false alarm level of a detection algorithm. We then define the change event of interest as follows:
where and are the smallest attack magnitudes of interest for and , respectively. Note that, in general, an attacker can arbitrarily choose its attack parameters, i.e., and do not restrict an attacker’s strategy. In fact, attackers usually do not know such parameters. On the other hand, smarter attackers may exploit such lower bounds on the attack magnitudes in order to perform stealthy attacks with small attack magnitudes (see Sec. IV-B).
The generalized CUSUM algorithm can then be written as follows:
where and denote the state estimates for the pre- and post-attack cases, respectively, is the decision statistic at time , is the test threshold, and is the generalized log-likelihood ratio (GLLR) calculated at time . Based on (16), the decision statistic can be recursively updated at each time as , where [38, Sec. 2.2].
Note that whenever reaches zero, the (generalized) CUSUM algorithm updates its change-point estimate to the current time , where the initial change-point estimate is [38, Sec. 2.2]. That is, when , we have . Recall that the Kalman filter for the post-attack case is employed assuming the normal measurement model (cf. (2)) up to the unknown change-point . We then propose to employ the Kalman filter for the post-attack case based on the estimated change-point . Hence, whenever the change-point estimate is updated, the Kalman filter for the post-attack case needs also to be updated. Recall further that the Kalman filter for the pre-attack case is always employed based on the normal measurement model. Hence, whenever , the Kalman filter estimates for the post-attack case are updated by setting them to the Kalman filter estimates for the pre-attack case, i.e., and .
Assuming no attack, is the optimal state estimate at time . Thus, we estimate by for the pre-attack case, i.e., . On the other hand, we estimate by for the post-attack case, i.e., . This is because the measurement update step of the Kalman filter for the post-attack case and hence depends on estimates of the unknown attack variables (cf. (12)), and effects of the attack parameters and at time on need to be blocked to be able to compute the MLEs of the attack parameters in closed form (cf. numerator in (16)). Note that is computed based on the measurements up to time , thus is independent of the attack parameters at time .
At first, it may seem unfair that we use the state estimate of the measurement update step, i.e., , for the pre-attack case, and the state prediction, i.e., , for the post-attack case. However, it essentially improves the performance of the proposed detection scheme due to the following reasons: (i) in case of no attack, we favor over and hence decrease the false alarm level of the proposed detection scheme, (ii) in case of an attack, since the state estimates for the post-attack case are recovered whereas the state estimates for the pre-attack case do not have a recovery mechanism, detection delays are not expected to increase.
Furthermore, based on (16), the following proposition presents the GLLR at time and the MLEs of the attack variables for the time interval between and .
Proposition 1: Let and . Moreover, let , , , and , , . The most likely subset of meters under no attack, under only FDI attack, under only jamming attack, and under both FDI and jamming attacks during the time interval between and are classified, respectively as
and the GLLR at time is computed as
Furthermore, the MLEs of the attack magnitudes for meter and for the interval between and are determined as follows:
Proof: See Appendix -A.
The proposed online detection and estimation algorithm is summarized in Algorithm 1. At each time , firstly the prediction step of the Kalman filters is implemented. Then, the most likely attack type (or no attack) and the attack parameters for each meter are determined. Based on the estimates of the attack variables, the measurement update step of the Kalman filters is implemented. Then, the GLLR is computed and the decision statistic is updated. If the decision statistic crosses the predetermined test threshold, then an attack is declared. Otherwise, it proceeds to the next time interval and further measurements are collected. Moreover, if the decision statistic reaches zero, the Kalman filter estimates for the post-attack case are updated before proceeding to the next time interval. Recall that Algorithm 1 keeps a change point estimate . Hence, after an attack is declared at time , to help for a quick system recovery, can be reported as the recovered state estimates and further, estimates of the attack types and the set of attacked meters can be reported for the time interval between and .
Remark 2: The detector parameters and can be determined by the system designer based on the system requirements, i.e., the desired level of false alarm rate. The system designer firstly determines the desired minimum level of average false alarm period, i.e., . If the frequency of false alarms needs to be decreased, then is chosen higher. After choosing , the system designer chooses the values of , , and the test threshold in order to achieve an average false alarm period that is larger than or equal to . For a higher level of , higher values of , , and need to be chosen. On the other hand, higher values of , , and lead to larger detection delays. Hence, the system designer can choose such parameters to strike a desired balance between false alarm rate and the detection delays.
Iv Stealthy Attacks and Countermeasures
We firstly discuss stealthy attacks against a CUSUM detector, which can be employed in a simple case where the pre- and post-attack pdfs are known. Discussion on the stealthy attacks against a CUSUM detector is useful since similar forms of stealthy attacks can be performed against all CUSUM-based detectors. We then particularly discuss stealthy attacks against the proposed detector, i.e., Algorithm 1, where the pre- and post-attack pdfs are unknown and time-varying, as explained in Sec. III. Finally, we present some countermeasures against the considered stealthy attacks.
Iv-a Stealthy Attacks Against a CUSUM Detector
Suppose the pre- and post-attack measurement pdfs are known and denoted with and , respectively such that for and for . In this case, the CUSUM algorithm is the optimum solution to (14) , given by
where denotes the stopping time, is the test threshold, is the decision statistic at time , and is the log-likelihood ratio (LLR) at time .
Iv-A1 Non-persistent attacks
The CUSUM algorithm is mainly designed for detecting persistent changes, i.e., it is assumed that an attack is launched at an unknown time and continued thereafter. It accumulates evidence (LLR) over time and declares a change (attack/anomaly) only if the accumulated evidence is reliably high (cf. (28)). Hence, with the purpose of increasing the detection delay of the CUSUM algorithm, a smart attacker can design an on-off attacking strategy to perform an intermittent (non-persistent) attack. That is, it can attack for a period of time, then wait for a period of time and repeat this procedure over its attacking period with the aim of keeping the decision statistic of the CUSUM algorithm, i.e., , below the decision threshold for so that the attack can be continued without being noticed.
Since the measurements are essentially random variables, an attacker cannot control the decision statistic deterministically; it can control it only on average. Note that attackers usually need simple and effective attacking strategies that require the minimum possible knowledge. Let denote the Kullback-Leibler (KL) divergence between and . The following proposition presents a simple necessary condition for an attacker, having the knowledge of and , to determine the on and off periods of a non-persistent stealthy attack against the CUSUM detector.
Proposition 2: Let be a threshold chosen by the attacker. The on and off periods have to be chosen as
in order to satisfy for , where and are positive integers denoting the on and off periods, respectively.
where the inequality is due to the fact that can take negative values in general ().
If , then
and if , then