I Introduction
Ia A Brief Overview of CyberAttacks and Countermeasures in Smart Grid
Due to the integration of advanced signal processing, communication, and control technologies, smart grid relies on a critical cyber infrastructure that is subject to adversarial cyber threats [1, 2, 3, 4]. The smart grid is regulated based on estimated system states and the main aim of attackers is to damage/mislead the state estimation mechanism and thereby to cause wrong/manipulated decisions in the energy management system of the smart grid. Some potential consequences of a successful cyberattack are regional power blackouts, manipulated electricity market prices [5, 6], and destabilization of the power grid [7]. Such cyberattacks are also seen in practice. For instance, on December 23, 2015, the Ukrainian power system was attacked and the resulting power blackout affected around 200,000 people for several hours [8].
The Ukraine attack has demonstrated that attackers have more capabilities than predicted [8]. Namely, (i) attackers can access and monitor the power system over long periods of time without being detected, (ii) attackers are able to perform cyberattacks by hacking smart grid components (smart meters, control centers, etc.), manipulating/jamming the network communication channels, and accessing and manipulating the database of the control center [8, 9, 2]. Hence, cyberattacks significantly threaten the safe and reliable operation of the power grid in practice. Effective countermeasures need to be developed considering the worstcase scenarios where the attackers are fully capable of performing a diverse range of cyberattacks. The first step in a defense mechanism is early detection of cyberattacks. After detecting an attack, effective mitigation schemes should then be implemented.
Recently, the false data injection (FDI) attacks [2, 10, 11, 12] and the jamming attacks [9, 13, 14, 15]
against the smart grid are extensively studied in the literature and several detectors are proposed. The proposed detectors are mostly outlier detectors, i.e., they classify a sample measurement as either normal or anomalous. Conventional detectors classify a measurement as anomalous if the measurement residual exceeds a certain threshold
[16, 10, 17, 18, 19]. More advanced machine learning techniques are also considered for classification of anomalous measurements
[20, 21]. Moreover, in [12], firstly a Markov graph model for system states is learned under normal system operation and then attacks/anomalies are detected based on the consistency of new measurements compared to the learned nominal model. Further, in [22], based on the least squares (LS) state estimator, a multistep procedure is presented to detect and classify cyberattacks on meter measurements, network line parameters, and network topology, and then to make corrections for attack mitigation.In [23, 24, 25], robust extended Kalman filters have been proposed where the main aim is to bound the effects of outliers on the state estimation mechanism. No specific attack types are considered so that using such schemes, it is not possible to distinguish a real attack from random outliers, e.g., due to heavytailed nonGaussian noise processes. Moreover, such schemes have breakdown points such that if outliers, significantly far away from the nominal measurements, are observed, then the proposed filters fail to keep track of the system state.
In order to improve the time resolution and also to detect cyberattacks more reliably, several online detectors based on the quickest detection theory are proposed. For instance, in [26] and [27], cumulative sum (CUSUM)based schemes are considered to detect FDI attacks where the state estimation is based on the conventional LS methods. More recently, in [28], CUSUMbased detection schemes are proposed to detect FDI and denial of service (DoS) attacks (separately) in a dynamic setting and their advantages over the outlier detectors and the LSbased detectors are demonstrated. Further, in [29], a nonparametric CUSUM detector is proposed that do not assume any attack model and only evaluates the deviation of meter measurements from the baseline statistics, i.e., normal system operation. In [30], a windowbased CUSUM detector is proposed for detection of FDI attacks where the attack parameters of interest are estimated based on the most recent sliding window of measurements.
IB Contributions
In this paper, we propose robust mechanisms for timely detection of potentially combined and stealthily designed FDI and jamming attacks. The proposed mechanisms are tightly connected to an estimation mechanism, which makes both the detection and state estimation schemes robust against unknown and timevarying attack variables. In particular, online maximum likelihood estimates (MLEs) of the attack types, set of attacked meters, and the attack magnitudes are used in attack detection. Moreover, recovered state estimates are computed based on the online MLE estimates of the attack variables. No restrictive assumptions are made about an attacker’s strategy, i.e., an attacker can design and perform arbitrarily combined FDI and jamming attacks, targeting any subset of meters in any magnitude and can also change its attack parameters over time. Further, considering the possibility of smarter and more capable attackers, additional countermeasures are proposed against stealthily designed cyberattacks. These make the proposed detection schemes highly robust against a significantly wide range of potential cyberattacks targeting the smart grid.
Since the smart grid is a highly complex network, any anomaly/failure in a part of the system can quickly spread over the network and lead to new unpredicted failures. Hence, timely attack detection and mitigation is crucial. In this paper, for timely detection, we present realtime detection mechanisms. Moreover, to help for timely attack mitigation and quick system recovery, we provide online estimates of the attack types, set of attacked meters and attack magnitudes. Note that having an estimate for the attack type can be useful since different countermeasures may need to be employed against different types of attacks. Further, considering that the real power grid is a huge network consisting of many meters, an estimate of the attacked meters can be critical for a timely and effective attack mitigation, e.g., via isolating the attacked meters during the recovery procedure. Moreover, estimates of attack magnitudes are needed to recover attackfree states.
We list our main contributions as follows:

A novel lowcomplexity online detection and estimation algorithm is proposed against (possibly) combined FDI and jamming attacks. The proposed algorithm is robust to unknown and timevarying attack types, magnitudes, and set of attacked meters. Further, recovered state estimates and closedform online MLE estimates of the attack variables are presented.

Stealthy attacks against CUSUMbased detectors and particularly against the proposed algorithm are introduced and analyzed.

Several countermeasures are proposed against the considered stealthy attacks.
IC Organization
The remainder of the paper is organized as follows. In Sec. II, the system model, attack models, state estimation mechanism, and the problem formulation are presented. In Sec. III, an online cyberattack detection and estimation algorithm is presented. In Sec. IV, stealthy attacks against CUSUMbased detectors are introduced and analyzed. Also, countermeasures against the considered stealthy attacks are presented. In Sec. V, the proposed detection schemes are evaluated extensively via simulations. Finally, the paper is concluded in Sec. VI
. Boldface letters denote vectors and matrices, and all vectors are column vectors.
Ii System Model and Problem Formulation
Iia System Model
The actual power grid is regulated based on a nonlinear AC power flow model [2]. On the other hand, the approximate linearized (around an operating point) DC power flow model is a good approximation that is widely used in the literature to describe the operation of the power grid [16, 10, 31]. Furthermore, static system model and consequently conventional static (LS) state estimation are not effective in capturing the dynamics of a power system due to timevarying load and power generation [11]. In addition, attack detection mechanisms based on static estimators are not effective in detecting timevarying cyberattacks and structured “stealth” FDI attacks [10], for which dynamic state estimatorbased detectors are known to be effective [28, 32].
We then model the power grid, consisting of buses and meters, as a discretetime linear dynamic system based on the commonly employed linear DC model [16, 10, 31] as follows:
(1)  
(2) 
where is the state vector denoting the phase angles of buses (one of the buses is considered as a reference bus), is the state transition matrix, is the process noise vector, is an identity matrix, and is the transpose operator. Further, is the vector consisting of meter measurements, is the measurement vector for meter , is the measurement matrix, is the measurement noise vector, and is the measurement noise vector for meter . Note that in each time interval between and , measurements are taken at each meter, where is usually small, and the collected measurements between and are processed at time . To increase the measurement redundancy against noise and also to estimate the unknown attack parameters more reliably in case of a cyberattack, needs to be chosen higher.
In general, the state transition and measurement matrices can also be dynamic. For instance, due to changes in network topology, i.e., on and off states of the switches and line breakers in the power grid, the measurement matrix may vary over time. In that case, instead of modeling the smart grid as a linear timeinvariant system as in (1) and (2), we can model it as a linear timevarying system where we can replace and by and , respectively. The results presented in this study can be generalized to the case of linear timevarying system model as long as and are known by the system controller at each time .
IiB Attack Models
We assume that at an unknown time , a cyberattack is launched to the system, where we particularly consider FDI attacks, jamming attacks, and their combination. The attack types, attack magnitudes, and the set of attacked meters can be timevarying. But, during a time interval, i.e., between and , we assume that the attack parameters stay constant. Next, we explain the attack models under consideration.
IiB1 FDI Attack
In case of an FDI attack, additive malicious data are injected into the measurements of a subset of meters. In practice, an FDI attack can be performed by manipulating the network communication channels or hacking meters and/or control centers in the smart grid [2, 8]. The measurement model in case of an FDI attack takes the following form:
(3) 
where denotes the injected false data at time . Since the attack magnitudes are assumed to be constant between and , for meter , , where is a vector consisting of s. Note that if meter is not under an FDI attack at time , then , otherwise .
IiB2 Jamming Attack
In case of a jamming attack, we assume that the attacker constantly emits additive white Gaussian noise (AWGN) to the network communication channels to compromise a subset of meter measurements. We consider jamming with AWGN since (i) it is a commonly employed jamming model in the literature [33, 34]
, (ii) it is a simple attacking strategy to perform, and (iii) in an additive noise channel with Gaussian input, for a given mean and variance, among all noise distributions, the Gaussian noise maximizes the mean squared error of estimating the channel input given the channel output
[35, 34]. Hence, an attacker can jam the communication channels with AWGN to maximize its damage on the state estimation mechanism.In case of a jamming attack, the measurement model can be written as follows:
(4) 
where denotes the jamming noise, , and where is the variance of the jamming noise targeting meter at time . If meter is not under a jamming attack at time , then , otherwise .
IiB3 Hybrid Attack
In case of a hybrid (combined) attack, FDI and jamming attacks are simultaneously launched to the system and hence the measurement model takes the following form:
(5) 
For meter under both FDI and jamming attacks at time , and . Since the FDI and jamming attacks can be considered as special cases of hybrid attacks, we consider (5) as the measurement model under the attacking regime, i.e., for .
Remark 1: If the noise terms in the normal system operation are AWGN (as in (1) and (2
)) and the jamming noise terms are mutually independent over the meters, then the considered hybrid FDI/jamming attacks span all possible data attacks. This is due to the fact that a Gaussian random variable is defined by its mean and variance, and through the hybrid attacks, mean and variance of the density of meter measurements can be arbitrarily changed (cf. (
5)). For instance, in case of a DoS attack, meter measurements are blocked and only a random or zero signal is received at the control center [9, 13, 14]. Hence, the DoS attack can be considered as a special case of the hybrid cyberattacks, i.e., a DoS attack can either be equivalent to an FDI attack with false data being in the same magnitude of the actual signal but with an opposite sign or a jamming attack with high level noise variances such that the actual signal can be neglected compared to the noise signal [28]. On the other hand, if the jamming noise is correlated over the meters or it is not normally distributed, then such an attack does not comply with the considered jamming attack model in (
4) and nor with (5). For such cases, we consider a nonparametric goodnessoffit test as a countermeasure (see Sec. IVC2).IiC Pre and PostAttack Measurement Densities
Let where is the measurement matrix for meter . Since the measurement matrix is determined based on the system topology, the rows of are identical, i.e., , where is a row of . Based on the considered postattack model in (5), a measurement obtained at meter during the time interval between and , i.e., can be written as
(6) 
where is the set of nonattacked meters, is the set of meters under only FDI attack, is the set of meters under only jamming attack, and is the set of meters under both FDI and jamming attacks at time . Note that , , , and are disjoint sets and .
Then, the probability density functions (pdfs) of the measurements in the pre and postattack regimes take respectively the following forms
:(7) 
and
(8) 
IiD State Estimation
Since the smart grid is modeled as a discretetime linear dynamic system with the Gaussian noise terms (cf. (1) and (2)), the Kalman filter is the optimal linear estimator in minimizing the mean squared state estimation error [36]. Further, since the measurement models for the pre and postattack periods are different (cf. (7) and (8)), two Kalman filters need to be simultaneously employed: one for assuming no attack occurs at all and one for assuming an attack occurs at an unknown time . Since the latter involves the unknown changepoint and the unknown attack parameters and , estimates of these unknowns are needed to employ the corresponding Kalman filter. As we will explain later, is estimated by the detection algorithm, and are estimated via the maximum likelihood (ML) estimation.
The Kalman filter is an iterative realtime estimator composed of prediction and measurement update steps at each iteration. Let the Kalman filter estimates for the pre and postattack cases be denoted with and , respectively where and for the prediction and measurement update steps at time , respectively. The Kalman filter equations at time are then given as follows:
Preattack – Prediction:
(9) 
Preattack – Measurement update:
(10) 
Postattack – Prediction:
(11) 
Postattack – Measurement update:
(12) 
where and denote the estimates of the state covariance matrix at time , and and denote the Kalman gain matrices at time for the pre and postattack cases, respectively. Note that the MLE estimates of the attack parameters are used in the measurement update step of the Kalman filter for the postattack case, where is the MLE of (cf. (26)) and is the MLE of (cf. (27)). Hence, and are, in fact, recovered state estimates in case of a cyberattack. Note, however, that ML estimation errors may lead to errors in computing the recovered state estimates.
IiE Problem Formulation
Our objective is detecting cyberattacks in a timely and reliable manner and the quickest detection theory [37, 38, 39] is well suited to this objective. In the quickest change detection problems, measurements become available sequentially over time and at each time, either a change is declared or further measurements are taken in the next time interval, where the aim is to optimally balance the detection delay and the false alarm rate. There are two main approaches in the quickest detection theory, namely Bayesian and nonBayesian. In a Bayesian setting, the change point is considered as a random variable with a known a priori distribution whereas in a nonBayesian setting, the change point is considered as nonrandom and unknown. Our problem better fits to the nonBayesian setting since we do not assume any a priori knowledge about the changepoint . Then, we consider the following objective function, proposed by Lorden [40]:
(13) 
where is the stopping time at which an attack is declared, denotes all measurements obtained up to time , and is the expectation under , that is the probability measure if the change occurs at time . Note that is called the worstcase average detection delay since it is maximized over the change point and also over all measurements obtained up to the changepoint. We then consider the following minimax optimization problem:
(14) 
where is called the average false alarm period, i.e., average stopping time when no change occurs at all (), and is a prespecified lower bound for .
Let the pre and postattack measurement pdfs given in (7) and (8) be denoted with and , respectively. Since the dynamic system state is not directly observed and the attack parameters and are completely determined by an attacker and hence unknown, both pdfs are unknown and timevarying. If the pre and postattack pdfs would be exactly known, then the wellknown CUSUM algorithm would be the optimal solution to (14) [41]. Nonetheless, the system state can be inferred using the Kalman filters and the MLEs of the unknown attack parameters can be computed. Then, following a generalized likelihood ratio approach [38, Sec. 5.3], [26, 28] and replacing the unknowns with their estimates, a generalized CUSUM algorithm can be used as a solution to (14).
In this paper, in addition to early attack detection, we also aim to recover the attackfree system states. Notice that in case of no attack, i.e., for , the Kalman filter for the preattack case (assuming no attack at all) is the optimal state estimator. However, after an attack occurs, the measurement model assumed in the preattack period (cf. (2)) is no longer true. Hence, the state estimates for the preattack case, i.e., and , deviate from the actual system state for . Recalling that an attack occurs at an unknown time and the measurements follow the postattack measurement model (cf. (5)) for , if the attack launch time and the attack magnitudes and would be exactly known, then the system state would be perfectly recovered for . Nonetheless, as we will explain more clearly in the next section, the (generalized) CUSUM algorithm always keeps a changepoint estimate in its memory and updates this estimate as the measurements become sequentially available over time [38, Sec. 2.2]. When an attack is declared at the stopping time , becomes the final changepoint estimate of the (generalized) CUSUM algorithm. Furthermore, the MLEs of the attack magnitudes, i.e., and , can be computed at each time . Then, employing a Kalman filter for the postattack case (cf. (11) and (12)) and computing the state estimates using the MLEs of the attack parameters in the measurement update step, recovered state estimates, i.e., and , can be obtained for .
Iii Online Attack Detection and Estimation
Since it is hard to distinguish noise from FDI/jamming attacks with small magnitudes, some minimum levels for the attack magnitudes need to be defined in order to control the false alarm level of a detection algorithm. We then define the change event of interest as follows:
(15) 
where and are the smallest attack magnitudes of interest for and , respectively. Note that, in general, an attacker can arbitrarily choose its attack parameters, i.e., and do not restrict an attacker’s strategy. In fact, attackers usually do not know such parameters. On the other hand, smarter attackers may exploit such lower bounds on the attack magnitudes in order to perform stealthy attacks with small attack magnitudes (see Sec. IVB).
The generalized CUSUM algorithm can then be written as follows:
(16) 
where and denote the state estimates for the pre and postattack cases, respectively, is the decision statistic at time , is the test threshold, and is the generalized loglikelihood ratio (GLLR) calculated at time . Based on (16), the decision statistic can be recursively updated at each time as , where [38, Sec. 2.2].
Note that whenever reaches zero, the (generalized) CUSUM algorithm updates its changepoint estimate to the current time , where the initial changepoint estimate is [38, Sec. 2.2]. That is, when , we have . Recall that the Kalman filter for the postattack case is employed assuming the normal measurement model (cf. (2)) up to the unknown changepoint . We then propose to employ the Kalman filter for the postattack case based on the estimated changepoint . Hence, whenever the changepoint estimate is updated, the Kalman filter for the postattack case needs also to be updated. Recall further that the Kalman filter for the preattack case is always employed based on the normal measurement model. Hence, whenever , the Kalman filter estimates for the postattack case are updated by setting them to the Kalman filter estimates for the preattack case, i.e., and .
Assuming no attack, is the optimal state estimate at time . Thus, we estimate by for the preattack case, i.e., . On the other hand, we estimate by for the postattack case, i.e., . This is because the measurement update step of the Kalman filter for the postattack case and hence depends on estimates of the unknown attack variables (cf. (12)), and effects of the attack parameters and at time on need to be blocked to be able to compute the MLEs of the attack parameters in closed form (cf. numerator in (16)). Note that is computed based on the measurements up to time , thus is independent of the attack parameters at time .
At first, it may seem unfair that we use the state estimate of the measurement update step, i.e., , for the preattack case, and the state prediction, i.e., , for the postattack case. However, it essentially improves the performance of the proposed detection scheme due to the following reasons: (i) in case of no attack, we favor over and hence decrease the false alarm level of the proposed detection scheme, (ii) in case of an attack, since the state estimates for the postattack case are recovered whereas the state estimates for the preattack case do not have a recovery mechanism, detection delays are not expected to increase.
Furthermore, based on (16), the following proposition presents the GLLR at time and the MLEs of the attack variables for the time interval between and .
Proposition 1: Let and . Moreover, let , , , and , , . The most likely subset of meters under no attack, under only FDI attack, under only jamming attack, and under both FDI and jamming attacks during the time interval between and are classified, respectively as
(17)  
(18)  
(19)  
(20) 
and the GLLR at time is computed as
(21) 
where
(22) 
(23) 
(24) 
and
(25) 
Furthermore, the MLEs of the attack magnitudes for meter and for the interval between and are determined as follows:
(26) 
and
(27) 
Proof: See Appendix A.
The proposed online detection and estimation algorithm is summarized in Algorithm 1. At each time , firstly the prediction step of the Kalman filters is implemented. Then, the most likely attack type (or no attack) and the attack parameters for each meter are determined. Based on the estimates of the attack variables, the measurement update step of the Kalman filters is implemented. Then, the GLLR is computed and the decision statistic is updated. If the decision statistic crosses the predetermined test threshold, then an attack is declared. Otherwise, it proceeds to the next time interval and further measurements are collected. Moreover, if the decision statistic reaches zero, the Kalman filter estimates for the postattack case are updated before proceeding to the next time interval. Recall that Algorithm 1 keeps a change point estimate . Hence, after an attack is declared at time , to help for a quick system recovery, can be reported as the recovered state estimates and further, estimates of the attack types and the set of attacked meters can be reported for the time interval between and .
Remark 2: The detector parameters and can be determined by the system designer based on the system requirements, i.e., the desired level of false alarm rate. The system designer firstly determines the desired minimum level of average false alarm period, i.e., . If the frequency of false alarms needs to be decreased, then is chosen higher. After choosing , the system designer chooses the values of , , and the test threshold in order to achieve an average false alarm period that is larger than or equal to . For a higher level of , higher values of , , and need to be chosen. On the other hand, higher values of , , and lead to larger detection delays. Hence, the system designer can choose such parameters to strike a desired balance between false alarm rate and the detection delays.
Iv Stealthy Attacks and Countermeasures
We firstly discuss stealthy attacks against a CUSUM detector, which can be employed in a simple case where the pre and postattack pdfs are known. Discussion on the stealthy attacks against a CUSUM detector is useful since similar forms of stealthy attacks can be performed against all CUSUMbased detectors. We then particularly discuss stealthy attacks against the proposed detector, i.e., Algorithm 1, where the pre and postattack pdfs are unknown and timevarying, as explained in Sec. III. Finally, we present some countermeasures against the considered stealthy attacks.
Iva Stealthy Attacks Against a CUSUM Detector
Suppose the pre and postattack measurement pdfs are known and denoted with and , respectively such that for and for . In this case, the CUSUM algorithm is the optimum solution to (14) [41], given by
(28) 
where denotes the stopping time, is the test threshold, is the decision statistic at time , and is the loglikelihood ratio (LLR) at time .
IvA1 Nonpersistent attacks
The CUSUM algorithm is mainly designed for detecting persistent changes, i.e., it is assumed that an attack is launched at an unknown time and continued thereafter. It accumulates evidence (LLR) over time and declares a change (attack/anomaly) only if the accumulated evidence is reliably high (cf. (28)). Hence, with the purpose of increasing the detection delay of the CUSUM algorithm, a smart attacker can design an onoff attacking strategy to perform an intermittent (nonpersistent) attack. That is, it can attack for a period of time, then wait for a period of time and repeat this procedure over its attacking period with the aim of keeping the decision statistic of the CUSUM algorithm, i.e., , below the decision threshold for so that the attack can be continued without being noticed.
Since the measurements are essentially random variables, an attacker cannot control the decision statistic deterministically; it can control it only on average. Note that attackers usually need simple and effective attacking strategies that require the minimum possible knowledge. Let denote the KullbackLeibler (KL) divergence between and . The following proposition presents a simple necessary condition for an attacker, having the knowledge of and , to determine the on and off periods of a nonpersistent stealthy attack against the CUSUM detector.
Proposition 2: Let be a threshold chosen by the attacker. The on and off periods have to be chosen as
in order to satisfy for , where and are positive integers denoting the on and off periods, respectively.
Proof.
We have
(29) 
where the inequality is due to the fact that can take negative values in general ().
If , then
and if , then
Let
Comments
There are no comments yet.