RelSen: An Optimization-based Framework for Simultaneous Sensor Reliability Monitoring and Process State Estimation

04/19/2020 ∙ by Cheng Feng, et al. ∙ Siemens AG 0

Recent advances in the Internet of Things (IoT) technology have led to a surge on the deployment of sensors everywhere. As a result, information obtained from sensors has become an important source for decision making in our daily life. Unfortunately, in most sensing applications, sensors are known to be error-prone due to various reasons and the measurements from sensors can also become unreliable at any unexpected time. Therefore, apart from the states of physical processes under measurement, we believe it is highly important to monitor the reliability of sensors as well. In this work, we propose RelSen: an optimization-based framework for simultaneous sensor reliability monitoring and process state estimation. In RelSen we formulate both the reliability scores of sensors and the states of monitored processes as variables to learn via an optimization problem only given the observed measurements from sensors. The optimization problem is efficiently solved by utilizing the redundancy of sensors, the correlation between multiple process states and the smoothness of each individual process. Furthermore, we apply RelSen for sensor reliability monitoring and process state estimation in an outdoor air pollution monitoring system and the condition monitoring system for a cement rotary kiln. Experimental results show that our framework can timely identify unreliable sensors and achieve higher accuracy than several baseline methods in process state estimation under three types of commonly observed sensor faults.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 11

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

With the trend of Internet of Things (IoT), sensors are becoming ubiquitous. The measurements from sensors have become an important source of knowledge to decision making for both human-beings and computing machines in different domains, such as industrial process control [18], air pollution monitoring [11] and moving object tracking [4]. Nevertheless, it is also well known that measurements from sensors can be error-prone due to various reasons such as hardware or software faults [22], extreme ambient environment conditions and even cyber attacks [30]

. This has motivated many researchers in both industrial and academic communities to craft specialized techniques to improve the dependability of decision making based on sensor measurements. To date, most techniques for mitigating the effects of sensor noise and errors can be broadly categorized into three classes: 1) state estimation methods which estimate the states of monitored processes by utilizing the prior knowledge about the process dynamics and the distribution of measurement noise. The family of Kalman filtering-based algorithms

[7, 8, 1] are within this class; 2) parameter estimation methods which estimate the parameters of the sensor noise that best describe the observed sensor measurements via the learned model of process state distributions, examples see [27, 28]

; 3) sensor fusion techniques which combine measurements from redundant sensors to achieve improved measuring accuracy than that could be achieved by the use of a single sensor alone. Typical fusion methods combine measurements from multiple sensors using mean, median or weighted average statistics based on known noise variance or covariance of sensors

[14]. Despite many successful applications of the above three techniques in the past decades, they all have certain limitations which restrict their applicability in the IoT context. For Kalman filtering-based state estimation methods, a system identification step [17] is often required to build up the mathematical models of the process dynamics before they can be applied. This step is however generally very challenging in practice. For parameter estimation methods, a model training phase is often required to capture the “normal” behaviour of the state distributions and/or transition dynamics of the underlying physical processes via models trained by a certain amount of observed data. As a result, their performances are likely to deteriorate when the process characteristics change which means that the trained models cannot well represent the process dynamics any more. For sensor fusion techniques, on one hand sensor redundancy is not necessarily available, on the other hand except the naïve mean and median method, how to assign weights to redundant sensors for fusion often becomes a state estimation or parameter estimation problem [14].

Surprisingly, despite the fact that sensors can be unreliable due to many reasons, to our knowledge, there has been very limited work focusing on enhancing the dependability of sensor systems via monitoring the reliability of sensors. We believe that monitoring the reliability of sensors can bring tremendous benefits. To name a few, sensor reliability provides an important metric for benchmarking between different sensor vendors, and customers knowing how reliable a sensor is can better decide whether to use or buy it. Furthermore, by monitoring the reliability of sensors, predictive maintenance of sensor systems can be conducted by timely identifying and replacing unreliable sensors. Most importantly, knowing the reliability of sensors can also improve the accuracy on the measurement of latent process states by giving less weights to unreliable sensors for estimating the states of monitored processes.

In this work, we propose RelSen, an optimization-based framework for simultaneous sensor reliability monitoring and process state estimation. In RelSen every sensor is assigned a reliability score that can be updated dynamically based on the latest behavior of the sensor. The reliability score of sensors are then utilized to better estimate the states of monitored physical processes. Specifically, we formulate both the reliability scores of sensors and the states of monitored processes as variables to learn via an optimization problem only given the observed measurements from sensors. The optimization problem is solved by an offline algorithm in the warm-up period which is required to learn the intrinsic reliability score of sensors without sensor faults. After the warm-up period, a more efficient online algorithm is used to dynamically update sensor reliability scores and estimate the monitored process states based on sensors’ behavior within a user-defined sliding window. In essence, RelSen can be considered as an advanced sensor fusion method in which fusion is conducted based on the principle that more reliable sensors should be more likely to provide sensor measurements which are closer to the states of the monitored physical processes, and the states of monitored physical processes should be closer to the measurements from more reliable sensors. The similar idea is also exploited in the information extraction field for truth discovery from multiple information sources [15]. Notably, in RelSen (hard) sensor redundancy is optional because soft sensors are automatically constructed which utilize the correlation between states of multiple processes to provide extra information sources for state estimation in the framework. Furthermore, both the offline and online algorithms in RelSen do not assume the model of process dynamics to be predefined, which means that the system identification step as in the Kalman filtering-based state estimation methods as well as the model training phase in the parameter estimation methods are also not required. This significantly improves the generality and applicability of RelSen in practice, especially in the IoT context where many processes with highly unpredictable dynamics need to be measured. To demonstrate its effectiveness, we apply RelSen for sensor reliability monitoring and process state estimation in two sensor systems: one deployed for outdoor air pollution monitoring, the other for condition monitoring of cyclones and decomposition furnaces in a cement rotary kiln. Experimental results show that our framework can timely identify unreliable sensors and outperform several baseline methods in process state estimation under three types of commonly observed sensor faults.

The remaining part of this paper is organized as follows. We first discuss the related work in the next Section. In Section III, we outline the research problem of this paper. This is followed by the formal introduction of RelSen, which includes the offline algorithm used in the warm-up period as presented in Section IV and the online algorithm used for real time sensor reliability monitoring and process state estimation as presented in Section V. Technical implementation issues of RelSen are discussed in Section VI. Then, the experiments on the outdoor air quality monitoring system and the cement rotary kiln condition monitoring system are presented in Section VII and VIII respectively. Finally, we draw the conclusion and discuss possible extensions in the last section.

Ii Related Work

Sensor reliability evaluation and process state estimation are generally regarded as two separate but closely related tasks. Sensor reliability or accuracy is commonly evaluated by comparing the sensor measurements with the ground truth of the monitored physical process states. However, as the ground truth of the measured process states is generally unknown, the accuracy of the sensor measurements can only be evaluated once the process states are estimated. Thus, state estimation, e.g., filtering algorithms [7, 2, 23, 8], parameter estimation, e.g., Bayesian estimation algorithms [27, 28] or sensor fusion techniques [25, 6, 14] have to be applied beforehand. To formulate the whole workflow, the authors of [24] proposed a sensor accuracy estimation framework which consists of four layers: pre-processing, state estimation, accuracy estimation and accuracy indexing. In their work, several taxonomies are proposed for the methods that can be used to implement process state estimation.

In RelSen, the two problems (sensor reliability monitoring and process state estimation) are jointly tackled in a single framework. Similarly, the authors of [31] also proposed a sensor reliability-based process state estimation method for environmental sensing applications. In their method named Influence Mean Cleaning (IMC), the reliability score of a sensor is incrementally updated by checking the distance between its measurement and the predicted true state of the underlying monitored process. The reliability score of a sensor with a distance smaller than a user-defined threshold will be increased, otherwise the reliability score will be decreased. The true state of the underlying monitored process is calculated as the sensor reliability-weighted mean of measurements from a group of spatially correlated sensors. Furthermore, the authors also discussed the effect of removing unreliable sensors on the accuracy of process state estimation in [32]. Compared with IMC, RelSen conducts sensor reliability-based process state estimation with an optimization-based framework which has better interpretability both intuitively and mathematically.

It is common that the same phenomenon can have many different views from different entities. To discover the truth from multiple data sources, entity reliability-based truth discovery has been studied for many years in the information retrieval domain [15]. Among the commonly used methods for truth discovery in information retrieval are voting-based methods [19, 20], optimization-based methods [12, 13] and probabilistic inference-based methods [33, 21, 16]. RelSen also takes an optimization-based method for process state estimation, however, the problem we face is more complex because we need to consider the evolving dynamics of the monitored processes, the unexpected faults from sensors and the processes whose states are only reported by a single data source (sensor).

Iii Problem Statement

In this section, we formulate the problem class which we study in this paper. We consider the general case where there are a number of sensors monitoring multiple physical processes in a system. Let be the set of physical processes, be the set of sensors in the system, we use to denote the set of sensors that are monitoring the physical process , where , and . That is to say, each physical process is monitored by one or more sensors, and each sensor can only monitor one physical process.

Let be the monitored signals from the sensors at a given discrete time , where the timestamps are a totally ordered set. Our target is to infer and , which are the quantified reliability scores of sensors and the true states of the monitored physical processes at time , respectively.

Iv Warm-up Period

Before monitoring the sensor reliability and estimating the true process states in real-time, a warm-up period is required in RelSen. During the warm-up period, we use an offline algorithm to infer the sensor reliability scores and the true states of the physical processes. Furthermore, let the warm-up period last for time steps, we assume no sensor faults occur during the period, thus the reliability score for each sensor is unchanged. Concretely, we use to denote the reliability scores of sensors within this period. Intuitively, we can imagine that reflects the intrinsic reliability of sensors in absence of faults. In the remainder of this section, the offline algorithm for deriving and will be presented.

Iv-a Initialization

In the initialization step of the offline algorithm, we set , , which means that we initialize the estimation of the true states for a physical process during the warm-up period as the mean value of the measurements from all sensors that are monitoring it. Note that the estimation will be updated in the subsequent steps.

Iv-B Random Local Linear Regression for Soft Sensor Construction

In order to enhance the inference of sensor reliability scores and true states of physical processes, for each physical process , besides the (hard) sensors, we further construct

soft sensors for monitoring its state at each time step by taking into account its correlation with other physical processes. The soft sensors are automatically constructed by random local linear regression. Specifically, the design of soft sensors based on local linear regression models is a popular method for process monitoring

[34, 9]. The difference between linear regression and local linear regression models is that local linear regression models are fitted only on a small subset of observations in a neighborhood which is similar to the new observation. As a result, local linear regression models are globally nonlinear, and often achieve better accuracy and can be promptly adapted when the process characteristics change.

In our context, we propose to use random local linear regression for constructing soft sensors. Specifically, when constructing a soft sensor, we select random subsets of sensors as explanatory variables to setup the local linear regression model. As a result, the soft sensors being constructed are weakly correlated with each other. The motivation of this randomness is similar to ensemble learning methods [29]: combining the predictions from a group of weak learners can form a strong learner; the combination in ensembles works better if the predictions from the sub-models are uncorrelated or at best weakly correlated.

Concretely, let be a target physical process. To build up a soft sensor for at time , we first randomly select sensors from the sensor set , where is a tunable ratio defined by the user. Let be the selected sensors which we call explanatory sensors for making a soft sensor for process at time ,

be the vector consists of the measurements from the explanatory sensors, we define a neighbor set

for the point . The neighbor set is derived by the K-nearest neighbors using Euclidean distance given the set of observed measurements in the warm-up period. Then the signal of the soft sensor for process at time is given by:

(1)

where , the weight of an explanatory sensor for constructing the soft sensor for process at time , takes the solution of the following optimization problem:

(2)

in which denotes the vector consisting of the measurements from the sensor set at time . Furthermore, we denote as the fitting error of the soft sensor, such that

where is the size of the neighbor set defined by the user.

Iv-C Optimization-based Framework for Sensor Reliability Scores and Process States Estimation

We propose to use an optimization-based framework to calculate the sensor reliability scores and estimate the true process states. The basic idea behind the framework is that more reliable sensors should be more likely to provide sensor measurements that are closer to the true states of the underlying physical processes, and the true states of monitored physical processes should be closer to the measurements from more reliable sensors. Concretely, our optimization framework is given as follows:

where is the normalized fitting error when constructing the soft sensor for process at time , such that in which ; works as the reliability score of the soft sensor for process at time , which means that the reliability score of a soft sensor is the weighted sum of the reliability scores of its explanatory sensors (an explanatory sensor with a larger absolute weight in constructing the soft sensor contributes a larger proportion of its reliability score to the soft sensor), scaled by the normalized fitting error when constructing the soft sensor (the higher fitting error, the lower reliability score).

The motivation behind the loss function

is as follows: 1) The term measures the weighted distance between measurements from the (hard and soft) sensors with the true states of the monitored processes, where the weights are the reliability scores of the sensors. By minimizing this part, a sensor that gives less accurate measurements will be assigned a lower reliability score; meanwhile, the estimated process states will be closer to the measurements from more reliable sensors. 2) The term is a smoothing factor enforcing the smoothness of estimated process states where

is a user-defined hyperparameter which controls the strength of the enforcement for process

. Note that here we use a simple smoothing factor as an illustration which forces two consecutive process states not to be far away from each other. In principle, other more complicated smoothing factors such as higher-order smoothing factors can also be applied in our framework. 3) The constraint term is required to make the optimization problem bounded. In addition, under this constraint term the solution of the optimization problem becomes easy to obtain as we will show later.

Since there are two sets of variables to be optimized, we apply the coordinate descent algorithm [26] to solve the constrained optimization problem. Specifically, we iteratively update sensor reliability scores and estimate process states in two steps until the distance between the estimated process states between two consecutive iterations is less than a threshold:

(3)

where denotes the estimated process states at time in the th iteration. The two steps in each iteration are as follows:

Iv-C1 Fix Process States, Update Sensor Reliability Scores

In this step, we assume process states are known, then the method of Lagrange multipliers is used to solve the optimization problem. Specifically, by introducing a Lagrange multiplier for the constraint, we obtain the following Lagrangian:

Since that the above function is convex (the sum of convex functions is also convex), the global optimum can be achieved by making partial derivative with respect to be zero:

where is an indicator function which equals 1 when the condition is satisfied and 0 otherwise. Rearranging the term to the r.h.s of the above equation, and since , we obtain:

Replacing back to Equation IV-C1, we finally obtain:

(5)

Iv-C2 Fix Sensor Reliability Scores, Update Process States

At this step, we assume the sensor reliability scores are known, and we update the process states to minimize the loss. Similarly, since the loss function is convex, the optimum estimation of process states can be obtained by making partial derivative with respect to to zero. Specifically, by setting , , it can be found that takes the solution of the following system of linear equations:

(6)

To summarize, we give the offline algorithm for sensor reliability and process state estimation in the warm-up period in Algorithm 1.

0:  , , , , , ,
1:  for   do
2:     for  do
3:        Initialize
4:     end for
5:  end for
6:  for   do
7:     for  do
8:        for  do
9:           Randomly select explanatory sensors from the sensor set
10:           Derive the -nearest neighbors for the point , where consists of measurements from the explanatory sensors
11:           Construct soft sensor as in Equation 1 by solving the optimization problem in Equation 2
12:        end for
13:     end for
14:  end for
15:  repeat
16:     Fix process states , update sensor reliability scores by Equation 5
17:     Fix sensor reliability scores , update estimated process states by solving the system of linear equations in Equation IV-C2
18:  until The convergence criteria in Equation 3 is satisfied
19:  return  ,
Algorithm 1 Sensor Reliability and Process State Estimation in the Warm-up Period

V Real-time Sensor Reliability Monitoring and Process State Estimation

In this section, we present an online algorithm for real-time sensor reliability monitoring and process state estimation which can be conducted after the warm-up period. The motivations are two-folds: 1) sensors can become unreliable at anytime due to various reasons, we shall dynamically update the reliability scores for sensors based on their latest behaviors. 2) the offline algorithm used in the warm-up period can be too costly for real-time usage since it needs to iteratively solve systems of linear equations until convergence.

Concretely, the online algorithm consists of three main steps, namely soft sensor construction, process state estimation and sensor reliability score update. The general idea behind the online algorithm is similar with the offline algorithm used in the warm-up period: constructing soft sensors for providing more information sources to the estimation of process states; the true states of monitored processes should be closer to measurements from information sources (hard and soft sensors) with higher reliability scores. The difference is that in the online algorithm, the reliability scores of sensors are dynamically updated at each time step based on a user-defined sliding window. Moreover, the algorithm is more efficient by utilizing the sensor reliability scores and process states that have been calculated in the previous time steps. In the remainder of this section, the details of the online algorithm will be described.

V-a Soft Sensor Construction

After the warm-up period, when new measurements are received at a time , for each process , we first construct soft sensors via random local linear regression as described in Section IV-B. Specifically, let be the measurements from the selected explanatory sensors for soft sensor , its neighbor set and the target values for fitting the local linear regression model is derived from the observed measurements and estimated process states from time interval .

V-B Process State Estimation

To estimate the process states at time , we propose the following simplified optimization framework:

in which and are constants that have been calculated in the previous time step. The intuition is that we use the sensor reliability scores at the previous time step to estimate the process states at the current time step.

Since is the only variable in the loss function, making the derivative with respect to be zero, we get a closed form solution:

(7)

V-C Sensor Reliability Score Update

After the process states are estimated, we can update the sensor reliability scores at time . Concretely, let be the length of a sliding window based on which the reliability scores of sensors are calculated, we only need to solve the following optimization problem:

Following the steps similar as in Section IV-C1, we can dynamically update the reliability score for each sensor based on the observed sensor measurements and the estimated process states within the sliding window as follows:

(8)

To sum up, we give the online algorithm for real-time sensor reliability monitoring and process state estimation in Algorithm 2.

0:  , , , , , , ,
1:  loop
2:     Receive new measurements from sensors
3:     for  do
4:        for  do
5:           Randomly select explanatory sensors from the sensor set
6:           Derive the -nearest neighbors for the point , where consists of measurements from the explanatory sensors
7:           Construct soft sensor as in Equation 1 by solving the optimization problem in Equation 2
8:        end for
9:     end for
10:     Estimate process states by Equation 7
11:     Calculate sensor reliability scores by Equation 8
12:     Emit and
13:  end loop
Algorithm 2 Real-Time Sensor Reliability Monitoring and Process State Estimation

Vi Implementation Issues

In this section, we discuss some implementation issues of RelSen. First of all, since the value range for the states of distinct physical processes can be rather different, our framework will tend to assign higher reliability scores to sensors which monitor physical processes with smaller value range if the raw measurements are used. Therefore, we suggest to normalize the values of measurements into range [0,1] for all sensors. Secondly, there are a few hyperparameters to set up before running the algorithms in our framework. We illustrate the trade-off by setting different values of these hyperparameters:

  • : With a larger value of , the sensor reliability scores in the warm-up period can be computed based on more data points, thus tend to be more accurate. However, it also increases the risk to include sensor faults in the period. This will violate the assumption that the sensor reliability scores are unchanged during the warm-up period. Furthermore, a larger value of also means a larger computational cost is required for the warm-up period.

  • : With a smaller value of , the derived soft sensors for a physical process will be less correlated, thus the possibility of having duplicated information sources for a physical process will be reduced. However, it will also increase the possibility of under-fitting for soft sensors, thus the soft sensors will provide less information in our optimization-based framework. A good way to tune is to apply cross validation on the data in the warm-up period.

  • : With a large value of , the neighborhoods may include too many training points that can result in regressions that oversmooth. Conversely, neighborhoods with too few points can result in regressions with incorrectly steep extrapolations [5]. can also be tuned by cross validation on the data in the warm-up period.

  • : We construct different number of soft sensors for different physical processes. A good principle to decide is that a monitored process with less responsible (hard) sensors shall generally have more soft sensors.

  • : As mentioned, the value of shall be decided by the prior knowledge about the smoothness of the monitored physical process.

  • : The value of shall be close to zero. Assume the values of sensor measurements are normalized, we explicitly set . We find that setting a smaller value will have limited impacts on the results.

  • : With a larger value of , more data points will be considered for updating sensor reliability scores, thus the computed reliability scores will be smoother and we can have higher confidence in identifying unreliable sensors. However, in the meantime the computing cost is also increased by considering more data points and the latency of identifying unreliable sensors may also be increased.

Vii Experiment on Sensors for Outdoor Air Pollution Monitoring

In this experiment, we apply RelSen for sensor reliability monitoring and process state estimation in an outdoor air pollution monitoring scenario. Specifically, in our experiment, 16 sensors are deployed to monitor 6 physical processes in a small area. The 6 monitored physical processes are the concentrations of NO, NO, PM, PM, CO and O in the air. Among the 16 deployed sensors, the number of sensors for monitoring each physical process is illustrated in Table I. Each sensor reports its measurement every hour. We collected measurements from the 16 sensors for four months.

Monitored physical processes NO NO PM PM CO O
Num. of responsible sensors 5 3 3 2 2 1
TABLE I: The sensor deployment schema in the outdoor air Pollution monitoring experiment

Since the true states of physical processes are unknown, it is difficult for us to evaluate the performance of our method. Thus, we consider injecting artificial sensor data faults to the collected data. Specifically, we consider three common types of sensor data faults which have been observed in real deployments as described in [22]: SHORT faults, NOISE faults and CONSTANT faults. For SHORT faults, there is a sharp change in the measurements between a single sensor from two successive points; For NOISE faults, the noise variance of the sensor increases within a number of successive data points; For CONSTANT faults, the sensor reports values with a constant offset for a number of successive data points. For evaluation purpose, we treat the mean value of measurements from the responsible sensors before fault injection as the ground truth of the monitored process state. In the implementation of our method, we set length of the warm-up period to seven days, thus . We further set , , and are set to values such that each process has five hard and soft sensors in total, are set to 1 for all processes. The length of sliding window is set to for experimental purpose.

Vii-a Baseline Methods

To demonstrate the benefits of our method, we further compare our performance with four baseline methods:

Vii-A1 Median

the MEDIAN method estimates a monitored process state as the median value of measurements from its responsible sensors. In case of only having two sensors, e.g., PM and CO in this experiment, the MEDIAN method will take the mean as the median.

Vii-A2 Mean

the MEAN method estimate a monitored process state as the mean value of measurements from its responsible sensors.

Vii-A3 Imc

the IMC method estimate a monitored process state as the weighted sum of the measurements from its responsible sensors, such that , where the weights are decided by the sensor reliability scores updated by the following rule:

and

where is the length of a sliding window, is an error threshold below which a sensor measurement is regarded as consistent with the estimated process state. We refer more details of the IMC method to [31]. Note that in case of no redundancy such as O in the experiment, the IMC method will deteriorate to reporting the measurements from the sensor without any processing. In our experiment, we set , is also set to for comparison.

Vii-A4 BayesGMM

BayesGMM is a parameter estimation method which employs a Bayesian framework such that , where and

are vectors of random variables representing sensor measurements and process states respectively,

is a Gaussian random vector with zero mean and diagonal covariance matrix . In the method, a training stage is required in which no sensor faults occur, and the distribution of

is initialized via the Gaussian mixture model (GMM):

where is the label for the th mixture component. During the monitoring stage, the process states are estimated as

where an Expectation-Maximization (EM) algorithm

[3] is used to estimate and simultaneously. More details of the BayesGMM method can be found in [27]. Note that the BayesGMM method does not explicitly utilize sensor redundancy for process estimation, as a result, let denote the estimated process state for sensor , we calculate . Furthermore, in the experiments, the length of the training stage is set to , and the number of GMM components is chosen by minimizing the Bayesian information criterion (BIC) score [10].

Vii-B Fault Injection

To check the performance of each method under different fault types, the experiment under each type of faults is conducted separately. Specifically, to inject a particular fault type, one responsible sensor for each physical process is selected as the faulty sensor. Furthermore, we evenly divide the data after the warm-up period into three stages, namely low, medium and high intensity stages. We inject faults with intensities in the three stages respectively.

To inject SHORT faults, we randomly pick of data points for each faulty sensor and replace with such that . To inject NOISE faults for a faulty sensor, we replace with with a random duration from 10 to 50 data points such that adjacent contaminated segments are 24 data points away from each other, where

is the standard deviation of sensor

in the data. CONSTANT faults are injected in a similar way with the NOISE faults, the only difference is that we replace with .

Vii-C Performance Evaluation

Fig. 1: Comparison of reliability score traces for sensors monitoring PM and O under SHORT faults
Fig. 2: Comparison of reliability score traces for sensors monitoring PM and O under NOISE faults
Fig. 3: Comparison of reliability score traces for sensors monitoring PM and O under CONSTANT faults

To illustrate the power of RelSen on timely identifying unreliable sensors, we compare the traces of reliability scores for sensors monitoring PM and O generated by RelSen and IMC under SHORT, NOISE and CONSTANT faults in Figure 1, 2 and 3 respectively. Due to lack of space, we select the PM and O sensors as representative cases for illustration. As shown in the figures, we can only identify the faulty PM sensor using reliability scores generated by the IMC method under NOISE and CONSTANT faults with low and medium intensities. This is because for the O sensor, the IMC method cannot update its reliability score without any available redundant sensors. For the PM sensors, even with redundant sensors, IMC still cannot identify the faulty sensor under SHORT faults because the method uses the proportion of identified abnormal measurements to set the reliability score, and thus its ability to detect sensors with SHORT faults (only appear one data point each time) is very limited. Furthermore, IMC is also very sensitive to the value of . Concretely, when the intensity of NOISE and CONSTANT faults is high, all sensors will be treated as unreliable sensors as the distance of their measurements to the weighted mean will all go beyond the threshold value. On the contrary, we can easily distinguish the faulty sensors from normal sensors based on the reliability scores generated by RelSen under all fault types. Specifically, we can clearly observe the fluctuation of reliability scores for the faulty sensors in the figures. For sensors monitoring PM, there is also an upward trend of reliability scores for the normal sensors in the figures. As the reliability scores in RelSen are evaluated relatively, the increasing of reliability scores for the normal sensors also reflects the unreliability of sensors without the same trend. For the sensor monitoring O, we can also observe a downward trend of its reliability score in all of the three figures, and the trend becomes more clear when the length of the sliding window becomes larger. This well indicates that RelSen can identify unreliable sensors even without redundancy.

Fig. 4: Accuracy comparison on process state estimation using different methods under SHORT faults
Fig. 5: Accuracy comparison on process state estimation using different methods under NOISE faults
Fig. 6: Accuracy comparison on process state estimation using different methods under CONSTANT faults

To show the performance of RelSen on process state estimation, we compare the mean absolute error (MAE) of the estimated process states by different methods under the three fault types in Figure 45 and 6 (the MAE is calculated using the normalized values). As shown in these figures, we can see that the MEDIAN method can only achieve good accuracy when there are more than three redundant sensors for the monitored process, i.e, the cases for NO, NO and PM. The performances of MEAN and IMC methods are much worse than RelSen for almost all cases. BayesGMM has comparable overall accuracy with RelSen under the three types of faults. However, we also find the performance of BayesGMM is very sensitive to the intensity of faults. Specifically, BayesGMM can achieve good performance with high intensity, but its relative performance becomes worse with smaller intensities where the noise level becomes less apparent. Compared with BayesGMM, the relative performance of RelSen compared with other methods is more stable under faults with different intensities. Furthermore, we find that although RelSen can achieve different accuracy with different values of , the difference is generally minor compared with the gap with other methods. To summarize, we can see that RelSen can achieve better accuracy than the MEAN and IMC methods, and more robust performance than the MEDIAN and BayesGMM methods on process state estimation under the three investigated fault types with different intensities.

Viii Experiment on Sensors for Condition Monitoring in a Cement Rotary Kiln

In this section, we present our results of experiment conducted on a sensor-based condition monitoring system in a cement rotary kiln. Specifically, in the system there are 20 sensors in total among which 16 monitor the temperature and negative pressure on the inlet and outlet cones of four cyclones deployed in parallel, one monitors the negative pressure on the outlet of a connected decomposition furnace, three monitor the temperature on the bottom, middle and top of the decomposition furnace. Each sensor reports its measurement every 30 seconds. We collected measurements from the 20 sensors for a week.

Fig. 7: The reliability scores of the faulty sensors generated by RelSen in the condition monitoring system of the cement rotary kiln
Fig. 8: Ground truth vs measured and estimated process states on faulty sensors in the condition monitoring system of the cement rotary kiln
Fig. 9: Accuracy comparison on process state estimation using BayesGMM and RelSen in the condition monitoring system of the cement rotary kiln

Since each sensor monitors a distant physical process (temperature and negative pressure on different parts in the system) in this experiment, the effects of MEDIAN, MEAN and IMC methods deteriorate to the same as the raw sensor measurements. Therefore, we mainly compare our process state estimation accuracy with BayesGMM. In the implementation of RelSen, we set length of the warm-up period to one day, thus . The BayesGMM also uses the first day’s data for training the GMM. We further set , , and . are set to 1 for all processes. The length of sliding window is set to .

Three sensors, namely the temperature sensor on the inlet cone of Cyclone 4, the temperature sensor on the outlet cone of Cyclone 1, and the negative pressure sensor on the outlet of the decomposition furnace are selected as the faulty sensors with SHORT, NOISE and CONSTANT fault injection respectively. All the three fault types are injected together after the warm-up period with intensity following the same equations in the previous experiment. For SHORT faults injection, of data points are contaminated. NOISE and CONSTANT faults are injected with a random duration of 240 to 360 data points such that adjacent contaminated segments are 120 data points away from each other.

In Figure 7, we show the reliability scores of the faulty sensors generated by RelSen in the experiment. We observe that the reliability scores of the three faulty sensors decrease steeply after the warm-up period. The reliability scores are stabilized to values which are significantly lower than their values in the warm-up period after a certain length of time period. This shows that we can quickly identify the three faulty sensors by monitoring their reliability scores.

In Figure 8, we present the raw measurements, the estimated states via BayesGMM and RelSen, and the ground truth of states for the monitored processes by the three faulty sensors. Furthermore, we compare the MAE of process state estimation from BayesGMM and RelSen for the three physical processes with faulty sensors in Figure 9. The result shows that RelSen can achieve higher accuracy than BayesGMM under all the three fault types. We believe the main reason behind this is that the performance of BayesGMM deteriorates dramatically when the process characteristics change during the monitoring stage. Consequently, the trained GMM cannot capture the normal behavior of the physical process any more. The evidence is clear in Figure 8, where BayesGMM fails to capture the dynamics of the physical processes around time points between 12000 to 13000. However, in RelSen we use random local linear regression for soft sensor construction, this allows the soft sensors to be promptly adapted to capture changing process characteristics, thus the estimated process states are more accurate.

Ix Conclusion

In this paper, we have proposed RelSen: a novel optimization-based framework for simultaneous sensor reliability monitoring and process state estimation in sensor systems. The main logic behind RelSen is fairly straightforward: more reliable sensors should provide more accurate measurements; the true states of monitored physical processes should be closer to the measurements from more reliable sensors. By utilizing the correlation between process states and sensor redundancy, RelSen can dynamically update the reliability scores of sensors and accurately estimate the monitored process states in real time only given the measurements from sensors after a warm-up period. In our experiments conducted respectively on sensor systems for outdoor air quality monitoring and cement rotary kiln condition monitoring, we demonstrated that RelSen can accurately and promptly identify unreliable sensors under three types of commonly observed sensor faults. Furthermore, we showed that RelSen outperformed several baseline methods regarding to process state estimation.

With less assumptions and knowledge requirements about the monitored process dynamics, we see potential for application of RelSen to a wide range of use-cases in the sensor-based IoT applications. In the future, we will study the effects of dynamically removing unreliable sensors on the accuracy of process state estimation theoretically, and seek to engineeringly improve the efficiency of RelSen.

References

  • [1] C. K. Chui, G. Chen, et al. (2017) Kalman filtering. Springer. Cited by: §I.
  • [2] P. Del Moral (1996) Non-linear filtering: interacting particle resolution. Markov processes and related fields 2 (4), pp. 555–581. Cited by: §II.
  • [3] A. P. Dempster, N. M. Laird, and D. B. Rubin (1977) Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological) 39 (1), pp. 1–22. Cited by: §VII-A4.
  • [4] K. Granström, M. Baum, and S. Reuter (2017) Extended object tracking: introduction, overview, and applications. Journal of Advances in Information Fusion 12 (2). Cited by: §I.
  • [5] M. R. Gupta, E. K. Garcia, and E. Chin (2008) Adaptive local linear regression with application to printer color management. IEEE Transactions on Image Processing 17 (6), pp. 936–945. Cited by: 3rd item.
  • [6] F. Gustafsson (2010) Statistical sensor fusion. Studentlitteratur. Cited by: §II.
  • [7] A. H. Jazwinski (1970) Stochastic processes and filtering theory. Vol. 64, Academic Press. Cited by: §I, §II.
  • [8] S. J. Julier and J. K. Uhlmann (2004) Unscented filtering and nonlinear estimation. Proceedings of the IEEE 92 (3), pp. 401–422. Cited by: §I, §II.
  • [9] M. Kano and K. Fujiwara (2012) Virtual sensing technology in process industries: trends and challenges revealed by recent industrial applications. Journal of chemical engineering of Japan, pp. 12we167. Cited by: §IV-B.
  • [10] C. Keribin (2000) Consistent estimation of the order of mixture models. Sankhyā: The Indian Journal of Statistics, Series A, pp. 49–66. Cited by: §VII-A4.
  • [11] P. Kumar, L. Morawska, C. Martani, G. Biskos, M. Neophytou, S. Di Sabatino, M. Bell, L. Norford, and R. Britter (2015) The rise of low-cost sensing for managing air pollution in cities. Environment international 75, pp. 199–205. Cited by: §I.
  • [12] Q. Li, Y. Li, J. Gao, L. Su, B. Zhao, M. Demirbas, W. Fan, and J. Han (2014) A confidence-aware approach for truth discovery on long-tail data. Proceedings of the VLDB Endowment 8 (4), pp. 425–436. Cited by: §II.
  • [13] Q. Li, Y. Li, J. Gao, B. Zhao, W. Fan, and J. Han (2014) Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data, pp. 1187–1198. Cited by: §II.
  • [14] W. Li, Z. Wang, G. Wei, L. Ma, J. Hu, and D. Ding (2015) A survey on multisensor fusion and consensus filtering for sensor networks. Discrete Dynamics in Nature and Society 2015. Cited by: §I, §II.
  • [15] Y. Li, J. Gao, C. Meng, Q. Li, L. Su, B. Zhao, W. Fan, and J. Han (2016) A survey on truth discovery. ACM Sigkdd Explorations Newsletter 17 (2), pp. 1–16. Cited by: §I, §II.
  • [16] Y. Li, Q. Li, J. Gao, L. Su, B. Zhao, W. Fan, and J. Han (2015) On the discovery of evolving truth. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 675–684. Cited by: §II.
  • [17] L. Ljung (1999) System identification. Wiley Encyclopedia of Electrical and Electronics Engineering, pp. 1–19. Cited by: §I.
  • [18] C. Lu, A. Saifullah, B. Li, M. Sha, H. Gonzalez, D. Gunatilaka, C. Wu, L. Nie, and Y. Chen (2015) Real-time wireless sensor-actuator networks for industrial cyber-physical systems. Proceedings of the IEEE 104 (5), pp. 1013–1024. Cited by: §I.
  • [19] J. Pasternack and D. Roth (2010) Knowing what to believe (when you already know something). In Proceedings of the 23rd International Conference on Computational Linguistics, pp. 877–885. Cited by: §II.
  • [20] J. Pasternack and D. Roth (2011) Making better informed trust decisions with generalized fact-finding. In

    Twenty-Second International Joint Conference on Artificial Intelligence

    ,
    Cited by: §II.
  • [21] J. Pasternack and D. Roth (2013) Latent credibility analysis. In Proceedings of the 22nd international conference on World Wide Web, pp. 1009–1020. Cited by: §II.
  • [22] A. B. Sharma, L. Golubchik, and R. Govindan (2010) Sensor faults: detection methods and prevalence in real-world datasets. ACM Transactions on Sensor Networks (TOSN) 6 (3), pp. 23. Cited by: §I, §VII.
  • [23] R. Van Der Merwe, A. Doucet, N. De Freitas, and E. A. Wan (2001) The unscented particle filter. In Advances in neural information processing systems, pp. 584–590. Cited by: §II.
  • [24] H. Wen, Z. Xiao, A. Markham, and N. Trigoni (2014) Accuracy estimation for sensor systems. IEEE Transactions on Mobile Computing 14 (7), pp. 1330–1343. Cited by: §II.
  • [25] Y. Wen, A. M. Agogino, and K. Goebel (2004) Fuzzy validation and fusion for wireless sensor networks. In ASME 2004 International Mechanical Engineering Congress and Exposition, pp. 727–732. Cited by: §II.
  • [26] S. J. Wright (2015) Coordinate descent algorithms. Mathematical Programming 151 (1), pp. 3–34. Cited by: §IV-C.
  • [27] C. Yuan and C. Neubauer (2007) Bayesian sensor estimation for machine condition monitoring. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, Vol. 2, pp. II–517. Cited by: §I, §II, §VII-A4.
  • [28] C. Yuan and C. Neubauer (2008) Robust sensor estimation using temporal information. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2077–2080. Cited by: §I, §II.
  • [29] C. Zhang and Y. Ma (2012)

    Ensemble machine learning: methods and applications

    .
    Springer. Cited by: §IV-B.
  • [30] J. Zhang, R. S. Blum, and L. Kaplan (2017) Cyber attacks on estimation sensor networks and iots: impact, mitigation and implications to unattacked systems. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3316–3320. Cited by: §I.
  • [31] Y. Zhang, C. Szabo, and Q. Z. Sheng (2014) Cleaning environmental sensing data streams based on individual sensor reliability. In International Conference on Web Information Systems Engineering, pp. 405–414. Cited by: §II, §VII-A3.
  • [32] Y. Zhang, C. Szabo, and Q. Z. Sheng (2016) Reduce or remove: individual sensor reliability profiling and data cleaning. Intelligent Data Analysis 20 (5), pp. 979–995. Cited by: §II.
  • [33] B. Zhao and J. Han (2012) A probabilistic model for estimating real-valued truth from conflicting sources. Proc. of QDB. Cited by: §II.
  • [34] Z. Zhu, F. Corona, A. Lendasse, R. Baratti, and J. A. Romagnoli (2011) Local linear regression for soft-sensor design with application to an industrial deethanizer. IFAC Proceedings Volumes 44 (1), pp. 2839–2844. Cited by: §IV-B.