# Minimum Error Entropy Kalman Filter

To date most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in the present paper we develop a new Kalman-type filter, called minimum error entropy Kalman filter (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC based KFs, the proposed filter is also an online algorithm with recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the minimum error entropy extended Kalman filter (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

## Authors

• 36 publications
• 1 publication
• 23 publications
• 69 publications
• 50 publications
09/15/2015

### Maximum Correntropy Kalman Filter

Traditional Kalman filter (KF) is derived under the well-known minimum m...
08/26/2016

### Maximum Correntropy Unscented Filter

The unscented transformation (UT) is an efficient method to solve the st...
09/07/2018

### Maximum Correntropy Derivative-Free Robust Kalman Filter and Smoother

We consider the problem of robust estimation involving filtering and smo...
08/07/2018

### Granger Causality Analysis Based on Quantized Minimum Error Entropy Criterion

Linear regression model (LRM) based on mean square error (MSE) criterion...
08/29/2019

### Revisiting the orbital tracking problem

Consider a space object in an orbit about the earth. An uncertain initia...
10/11/2017

### Quantized Minimum Error Entropy Criterion

Comparing with traditional learning criteria, such as mean square error ...
09/04/2018

### Equalization with Expectation Propagation at Smoothing Level

In this paper we propose a smoothing turbo equalizer based on the expect...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Kalman filtering is a powerful technology for estimating the states of a dynamic system, which finds applications in many areas including navigation, guidance, data integration, pattern recognition, tracking and control systems

[1, 2, 3, 4, 5]. The original Kalman filter (KF) was derived for a linear state space model with Gaussian assumption [6, 2]. To cope with nonlinear estimation problems, a variety of nonlinear extensions of the original Kalman filter have been proposed in the literature, including extended Kalman filter (EKF) [7, 8], unscented Kalman filter (UKF) [9], cubature Kalman filter (CKF) [10] and many others. However, most of these Kalman filters are developed based on the popular minimum mean square error (MMSE) criterion and will face performance degradation in case of complicated noises, since in general MMSE is not a good choice for the estimation in non-Gaussian noises.

In recent years, to solve the performance degradation problem in heavy-tailed (or impulsive) non-Gaussian noises, some robust Kalman filters have been developed by using certain non-MMSE criterion as the optimality criterion [11, 12]. Particularly, the maximum correntropy criterion (MCC) [13, 14] in information theoretic learning (ITL) [11, 12] has been successfully applied in Kalman filtering to improve the robustness against impulsive noises. Typical examples include the maximum correntropy based Kalman filters [15, 16, 17, 18, 19, 20, 21, 22, 23, 24], maximum correntropy based extended Kalman filters [25, 26, 27], maximum correntropy based unscented Kalman filters [28, 29, 30], maximum correntropy based square-root cubature Kalman filters [31, 32]

and so on. Since correntropy is a local similarity measure and insensitive to large errors, these MCC based filters are little influenced by large outliers

[13, 33].

The MCC is a nice choice for dealing with heavy-tailed non-Gaussian noises, but its performance may not be good when facing more complicated non-Gaussian noises, such as noises from multimodal distributions. The minimum error entropy (MEE) criterion [34, 35] is another important learning criterion in ITL, which has been successfully applied in robust regression, classification, system identification and adaptive filtering [34, 35, 36, 37, 38]. Numerous experimental results show that MEE can outperform MCC in many situations although its computational complexity is a little higher [39, 40]. In addition, the superior performance and robustness of MEE have been proved in [41]. The goal of this work is to develop a new Kalman-type filter, called minimum error entropy Kalman filter (MEE-KF), by using the MEE as the optimality criterion. The proposed filter uses the propagation equations to obtain the prior estimates of the state and covariance matrix, and a fixed-point algorithm to update the posterior estimates and covariance matrix, recursively and online. To further improve the performance in the nonlinear situations, the MEE criterion is also incorporated into EKF, resulting in minimum error entropy extended Kalman filter (MEE-EKF).

The rest of the paper is organized as follows. In section II, we briefly review the KF algorithm and MEE criterion. In section III, we develop the MEE-KF algorithm. Sections IV and V provide the computational complexity and convergence analysis, respectively. In section VI, the MEE-EKF is developed. The experimental results are presented in section VII and finally, the conclusion is given in section VIII.

## 2 Background

### 2.1 Kalman Filter

Consider a linear dynamic system with unknown state vector

and available measurement vector . To estimate the state , Kalman filter (KF) assumes a state space model described by the following state and measurement equations:

 x(k)=Fx(k−1)+q(k−1) (1) y(k)=Hx(k)+r(k), (2)

where and are the state-transition matrix and measurement matrix, respectively. Here, the process noise and measurement noise are mutually independent, and satisfy

 E[q(k−1)]=0;E[r(k)]=0 (3) E[q(k−1)qT(k−1)]=Q(k−1) (4) E[r(k)rT(k)]=R(k), (5)

where and are the covariance matrices of and , respectively. In general, the KF includes two steps:

(1) Predict: The a-priori estimate and the corresponding error covariance matrix are calculated by

 ^x(k|k−1)=F^x(k−1) (6) P(k|k−1)=FP(k−1)FT+Q(k−1). (7)

(2) Update: The a-posteriori estimate and the corresponding error covariance matrix are obtained by

 ^x(k)=^x(k|k−1)+K(k)[y(k)−H^x(k|k−1)] (8) K(k)=P(k|k−1)HT[HP(k|k−1)HT+R(k)]−1 (9) P(k)=[I−K(k)H]P(k|k−1)[I−K(k)H]T +K(k)R(k)KT(k), (10)

where is the Kalman filter gain.

### 2.2 Minimum Error Entropy Criterion

Different from the MMSE [34] and MCC [35], the MEE aims to minimize the information contained in the error. In MEE, the error information can be measured by the Renyi’s entropy:

 Hα(e)=11−αlogVα(e), (11)

where is the order of Renyi’s entropy, and denotes the information potential defined by

 Vα(e)=∫pα(x)dx=E[pα−1(e)], (12)

where

is the probability density function (PDF) of error

and denotes the expectation operator. In practical applications, the PDF can be estimated by Parzen’s window approach [11]:

 ^p(x)=1NN∑i=1Gσ(x−ei), (13)

where denotes the Gaussian kernel with kernel size ; are error samples. Combining (12) and (13), one can obtain an estimate of the second order () information potential :

 ^V2(e)=1NN∑i=1^p(ei)=1N2N∑i=1N∑j=1Gσ(ei−ej). (14)

Since the negative logarithmic function is monotonically decreasing, minimizing the error entropy means maximizing the information potential .

## 3 Minimum Error Entropy Kalman Filter

### 3.1 Augmented Model

First, we denote the state prediction error as

 ϵ(k|k−1)=x(k)−^x(k|k−1). (15)

Combining the above state prediction error with the measurement equation (2), one can obtain an augmented model

 [^x(k|k−1)y(k)]=[InH]x(k)+μ(k), (16)

where denotes a identity matrix, and

 μ(k)=[−ϵ(k|k−1)r(k)] (17)

is the augmented noise vector comprising of the state and measurement errors. Assuming that the covariance matrix of the augmented noise is positive definite, we have

 E[μ(k)μ(k)T]=Θ(k)Θ(k)T=[Θp(k|k−1)Θp(k|k−1)T00Θr(k)Θr(k)T], (18)

where , and are obtained by the Cholesky decomposition of , and , respectively. Multiplying both sides of (16) by gives

 d(k) =W(k)x(k)+e(k), (19)

where

 (20) W(k)=Θ−1(k)[lInH] (21) (22)

with , , and .

### 3.2 Derivation of MEE-KF

Based on (14), the cost function of MEE-KF is given by

 JL(x(k))=1L2L∑i=1L∑j=1Gσ(ej(k)−ei(k)). (23)

Then, the optimal solution to is achieved by maximizing the cost function (23), that is

 ^x(k)=argmaxx(k)JL(x(k)) =argmaxx(k)1L2L∑i=1L∑j=1Gσ(ej(k)−ei(k)). (24)

Setting the gradient of the cost function regarding to zero, we have

 ∂∂x(k)JL(x(k)) =1L2σ2L∑i=1L∑j=1([ej(k)−ei(k)]Gσ(ej(k)−ei(k))[wj(k)−wi(k)]) =Γ1−Γ2−Γ3+Γ4=2Γ1−2Γ3 =2L2σ2W(k)TΨ(k)e(k)−2L2σ2W(k)TΦ(k)e(k) =0, (25)

where

 Γ1=1L2σ2L∑i=1L∑j=1ej(k)Gσ(ej(k)−ei(k))% wj(k)T (26) Γ2=1L2σ2L∑i=1L∑j=1ei(k)Gσ(ej(k)−ei(k))% wj(k)T (27) Γ3=1L2σ2L∑i=1L∑j=1ej(k)Gσ(ej(k)−ei(k))% wi(k)T (28) Γ4=1L2σ2L∑i=1L∑j=1ei(k)Gσ(ej(k)−ei(k))% wi(k)T (29) [Φ(k)]ij=Gσ(ej(k)−ei(k)) (30) [Ψ(k)]ij=L∑i=1Gσ(ej(k)−ei(k)). (31)

From (3.2), can be solved by a fixed-point iteration:

 ^x(k) =g(^x(k)) =(W(k)TΛ(k)W(k))−1(W(k)TΛ(k)d(k)), (32)

where

 Λ(k)=Ψ(k)−Φ(k)=[Λx(k)Λyx(k)Λxy(k)Λy(k)], (33)

where . The explicit expressions of , , and are

 Λx(k)=(Λi,j(k))n×n=(Φi,j(k))n×n−(Ψi,j(k))n×n (i=1,2⋯n;j=1,2⋯n) (34) Λxy(k)=(Λi,j(k))m×n=(Φi,j(k))m×n−(Ψi,j(k))m×n (i=n+1,n+2⋯n+m;j=1,2⋯n) (35) Λyx(k)=(Λi,j(k))n×m=(Φi,j(k))n×m−(Ψi,j(k))n×m (i=1,2⋯n;j=n+1,n+2⋯n+m) (36) Λy(k)=(Λi,j(k))m×m=(Φi,j(k))m×m−(Ψi,j(k))m×m (i=n+1,n+2⋯n+m; j=n+1,n+2⋯n+m). (37)

According to Eqs. (20), (21) and (33), we arrive at

 (W(k)TΛ(k)W(k))=[Θ−1p(k|k−1)TΛx+HTΘ−1r(k)TΛxy]Θ−1p(k|k−1)+[Θ−1p(k|k−1)TΛyx+HTΘ−1r(k)TΛy]Θ−1r(k)H(W(k)TΛ(k)D% (k))=[Θ−1p(k|k−1)TΛx+HTΘ−1r(k)TΛxy]Θ−1p(k|k−1)^x(k|k−1)+[Θ−1p(k|k−1)TΛyx+HTΘ−1r(k)TΛy]Θ−1r(k)y(k). (38)

By (38), the Eq. (3.2) can be rewritten as

 ^x(k) =(Ω1+Ω2Ω3)−1(Ω1^x(k|k−1)+Ω2y(k)) =(Ω1+Ω2Ω3)−1Ω1^x(k|k−1) +(Ω1+Ω2Ω3)−1Ω2y(k), (39)

where

 Ω1=[Θ−1p(k|k−1)TΛx+HTΘ−1r(k)TΛxy]Θ−1p(k|k−1)Ω2=[Θ−1p(k|k−1)TΛyx+HTΘ−1r(k)TΛy]Θ−1r(k)Ω3=H.

By using the matrix inversion lemma

 (A+BCD)−1=A−1−A−1B(C−1+DA−1B)−1DA−1, (40)

with the identifications

 Ω1→A,Ω2→B,IL→C,Ω3→D, (41)

one can reformulate (3.2) as

 ^x(k)=^x(k|k−1)+¯¯¯K(k)(y(k)−H^x(k|k−1)), (42)

where

 ¯¯¯K(k)=⎡⎢⎣¯¯¯P(k|k−1)+HT¯¯¯Pxy(k|k−1)+(¯¯¯Pyx(k|k−1)+HT¯¯¯R(k))H⎤⎥⎦−1×(¯¯¯Pyx(k|k−1)+HT¯¯¯R(k))¯¯¯P(k|k−1)=Θ−1p(k|k−1)TΛx(k)Θ−1p(k|k−1)¯¯¯Pxy(k|k−1)=Θ−1r(k)TΛxy(k)Θ−1p(k|k−1)¯¯¯Pyx(k|k−1)=Θ−1p(k|k−1)% TΛyx(k)Θ−1r(k)¯¯¯R(k)=Θ−1r(k)TΛy(k)Θ−1r(k). (43)

Then, the posterior covariance matrix can be updated by

 P(k)= [I−¯¯¯K(k)H]%P(k|k−1)[I−¯¯¯K(k)H]T +¯¯¯K(k)R(k)¯¯¯K(k)T. (44)

With the above derivations, the proposed MEE-KF algorithm can be summarized as Algorithm 1.

## 4 Computational Complexity

This section provides the comparison of the computational complexities of KF, maximum correntropy Kalman filter (MCKF) [15] and MEE-KF in terms of the floating point operations.

The KF updates with Eqs. (6)-(10), and the corresponding floating point operations are given in Table 1. From Table 1, we can conclude that the computational complexity of KF is

 CKF=8n3+10n2m−n2+6nm2−n+O(m3). (55)

According to [15], the computational complexity of MCKF is

 CMCKF =(2T+8)n3+(4T+6)n2m+(2T−1)n2 +(4T+2)nm2+(3T−1)nm+(4T−1)n +2Tm3+2Tm+TO(n3)+2TO(m3), (56)

where denotes the fixed-point iteration number, which is relatively small in general as shown in simulations in Section VII.

The updates of MEE-KF involve Eqs. (6), (7), (45)-(52) and (54), and the corresponding floating point operations are shown in Table 1. According to Table 1, the computational complexity of MEE-KF is

 CMEE−KF =(7T+8)n3+(7T)m3+(19T+6)n2m−n2 +(15T+2)nm2+TO(n3)+TO(mn) +Tm+(5T−1)n+(7T−1)nm +TO(m2)+2TO(m3). (57)

The MEE-KF has an additional computational burden induced by the error entropy functions in comparison to KF, and has a slightly higher computational complexity than MCKF. In the sense of order of magnitude, the computational complexities of the MEE-KF, MCKF and KF have no significant difference.

## 5 Convergence Issue

This section provides a sufficient condition to ensure the convergence of the fixed point iterations in MEE-KF, where the proof is similar to [35] and thus will not be provided here.

First, from Eq. (3.2), we can rewrite

 g(x(k))=M−1ww% Mdw (58)

with and .

Thus, a Jacobian matrix of with respect to gives

 ∇x(k)g(x(k))=∂∂x(k)M−1wwMdw=[∂g(x(k))∂x1(k)∂g(x(k))∂%x2(k)⋯∂g(x(k))∂xL]=−M−1ww2L2σ2(L∑i=1L∑j=1ξ1(k)[%wj(k)−wi(k)]×[wj(k)−wi(k)]T)g(x(k))+M−1ww2L2σ2(L∑i=1L∑j=1ξ1(k)[dj(k)−di(k)]×[wj(k)−wi(k)]), (59)

with