I Introduction
Processing Electronic Support Measure (ESM) data in surveillance systems [14, 20, 9] is one of the fundamental tasks in Recognition and Identification (R&I) [18, 16]. Identifying the emitter type based on the signals received from an ESM sensor has been addressed in the literature [11, 24, 16]
. The approaches in these previous works are suitable for systems with a single ESM sensor. In modern surveillance systems, the combination of radar reports and ESM is used to identify the targets and to further classify them into known platforms
[8, 12, 17]. These systems benefit from the features that can be extracted from the kinematic [2] of the targets as well as those of from ESM to classify the targets with higher accuracy. With the advances in recent years, the number of sensors has multiplied in modern surveillance systems and the need for new sensor fusion architectures has been identified. In [13], a comprehensive categorization of possible levels of processing that can be utilized in different applications such as air–to–ground and ground–to–ground surveillance was given.The primary focus of this paper is to handle multiple ESM sensors along with radar (kinematic) measurements to improve R&I accuracy. Previous works have addressed design of algorithms for processing and fusing ESM data in different contexts. In [25]
, ESM sensors and their capabilities are investigated for air defence system design. The concept of attribute estimation is discussed for multisensor scenarios in
[3]. Going beyond the traditional architectures for R&I systems, in this paper, a new approach for processing ESM and kinematic data is proposed to achieve the goals introduced in standards such as STANAG [26]. Although the Bayesian framework [4] is mainly considered in most R&I systems, this paper considers other frameworks such as evidence theory [31] and possibility theory and fuzzy sets [30], whenever possible, in addition to probabilistic solutions in different blocks of the new hierarchy, to achieve a better utilization of available information.If multiple ESM sensors are available in a surveillance system, there may be a variety of information in their output. For example the ESM data sent to the fusion center may range from raw (unprocessed) ESM data to fully processed information such as threat category or lethality [15]
of a specific target. Some of these data can be considered as probabilistic distributions while others may be in the form of sets and intervals possibly with probabilities assigned to them. The fusion of heterogeneous data for R&I and the management of the flow of information in the scenarios is the focus of this paper. This development motivates the new architecture for fusing all information to better identify the targets.
In order to understand the problem, the traditional frameworks that has been used to process ESM data are reviewed in Section II. In Section III, the fusion of data from a single ESM and radar sensor is considered. Section IV is devoted to the new system design and to show how it can be used in modern ESM fusion systems. In Section V, a simple example is considered with discussions on possible issues in the design of the proposed architecture. Finally, conclusions and future works are discussed in Section VI.
Ii Review of frameworks for R&I
There are different R&I frameworks one can use in surveillance systems. In advanced systems, the ability to take advantage of different frameworks is advantageous. Due to this emergence of multiple fusion approaches and in order to further discuss and formulate the ideas that are necessary for the development of the new architecture proposed in this paper, a brief review of common fusion frameworks for R&I is given in this section.
Iia Probability theory – Bayesian framework
Bayesian framework refers to the classical method that uses the Bayes’ theorem in statistical inference, which includes testing hypotheses and deriving estimates. The core of the Bayesian framework is to update the probability of a hypothesis or the probability distribution of a random variable when evidence (measurement) becomes available based on the Bayes’ formula
[2]:(1)  
for hypothesis (event) and
(2)  
for random variables. In the above, and
are the posterior probabilities and,
andare the prior probabilities. Bayesian framework is particularly useful when evidence is received sequentially. In this case, the posterior probability becomes the next prior probability at the next time instant.
IiB Evidence theory – Belief functions
In probability distributions, beliefs are distributed over the elements of the outcome space. In contrast, for a belief function, the distribution, also known as mass function, attributes belief to subsets of the outcome space. In other words, if there is belief attributed to a subset, there is a reason to believe that the outcome will be among the elements of that subset, without committing to any preference among those elements. One can define a mass distribution as a mapping from subsets of a frame of discernment into the unit interval:
(3) 
such that
(4) 
In this context, focal elements are defined as subsets that are attributed to non–zero mass. It is worth noting that the belief in a hypothesis is constrained to lie within an interval , where
(5) 
These bounds are known as belief (support) and plausibility, respectively. To combine two different belief functions together, one can use the Dempster’s rule of combination [29] as the fusion operator. Note that this rule finds the common shared belief between different sources and ignores all the conflicting belief through a normalization factor. Therefore, the combination (joint mass) of two sets of masses and can be written as
(6)  
(7)  
where is the measure of the amount of conflict between the two mass sets and is defined by
(8) 
IiC Possibility and fuzzy set theory
To further process the information that has linguistic characteristics, the idea of fuzzy sets and possibility theory can be used [30]. If possibility distributions are defined as fuzzy restrictions, then the imprecision in natural languages, which is possibilistic rather than probabilistic, can be addressed in a systematic manner. This subject is beyond the scope of this paper and will be explained in the work in progress for a general framework of ESM data fusion.
Iii Problem Formulation
Classification of targets into different categories is one of the most important tasks in Recognition and Identification. There are works in literature on different ways for the classification of data in distributed tracking systems for the purpose of recognition and identification [5, 7]. In this paper we focus on addressing the issue of the fusion of ESM and kinematic measurements at three different levels. Figure 1 shows a general block digram on how the classification process is carried out based on ESM and kinematic measurements.
For air–to–ground target tracking, if altimeter or terrain data is available, an estimate of the ground or sea level targets can be found by using a single ESM sensor alone [10]. This estimate has the advantage of being uncorrelated with data processed for estimating track features and further can be fused with kinematic estimate of other sensors.
In Figure 1, output types can be numerical, objective, structures, low–level (data), etc., assuming that they are hard data. Due to the existence of various types of data, such data should be defined within the proposed framework so that each block can use the data it receives. If soft data such as operator reports are available, qualitative, subjective, unstructured and higherlevel (information) data also must be included within the framework. Imperfect nature of information can also be considered in the architecture. These imperfections in data can come from different sources such as the uncertainty in the information, imprecision and the conflict between the processed data.
Iiia ESM measurement
Electronic support measures consist of various measurements and depend on the sensors that are mounted on the platform. To give a better view of the problem, in this paper, it is assumed that ESM measurements are Pulse Repetition Interval (PRI) high, PRI low, frequency high, frequency low, Pulse Width (PW) high, PW low and amplitude. Table I gives a short description of these measurements.
ESM mseasuement  Mean 

PRI High  
PRI Low  
Frequency High  
Frequency Low  
PW High  
PW Low  
Amplitude 
All these measurements are perturbed with white noise of known distributions with means defined in Table
I. Measured sensor data can be sent to the next block for further processing via a vector and its associated distribution model parameters as
.IiiB Attribute estimate
Target attributes are various variables with discrete values that primarily give information to distinguish between the targets by characterizing the measurements into different groups by assigning probabilities to possible outcomes [3]. Assuming
is received, the measurement can be characterized by the conditional probability density function (pdf) for
attributes as(9) 
where is the likelihood function of the attribute . In addition, prior information of the attributes can be modeled as
(10) 
Starting with processing the first measurement, the conditional probability of the target having attribute is, in the Bayesian framework
(11) 
where the normalization factor is
(12) 
If more measurements are available through different time steps, the recursive updatimg of attribute can be written as
(13) 
where
(14) 
Note that it is assumed the measurements are independent conditioned on the attribute. Then, the so called attribute estimate can be shown as a list of probabilities assigned to different attributes. Based on practical surveillance systems, informative attributes with possible outputs are shown in Table II. Each probability or evidence data presented with can be a list of scalars or intervals with their associated probabilities or probability density functions. The choice of the data type depends on the application and model of the sensor.
Attribute  

Speed  
Shape  
Length  
Size  
Emitter ID  
Number of emitters 
IiiC Recognition of attributes estimates
In order to further process the data received at this stage, i.e., , it is necessary to use a framework that can combine different types of information in . For example, to find out the threat category of a target, which can be friendly or foe, one should combine the information of the shape and speed together. If this information is in the form of lists, one can use belief functions and combination rules to fuse and later classify the data [31, 29, 21, 22, 23]. If all the data is in terms of probabilistic models, Bayesian framework can be adopted for the purpose of classification [28]. In the case of a combination of belief and probabilistic functions, approximations can be applied to belief functions as stated in [27] to be compatible with the Bayesian framework. In all cases, it is important to design the model such that uncertainty information is also generated in addition to recognition and classification results.
IiiD Position measurement
In tracking applications, the target position is mainly measured by a radar or sonar, in polar or spherical coordinates [2, 1]. However, target motion is usually best modeled in Cartesian coordinates. To cope with different coordinates used in measurement and target motion space, the received measurement is converted into Cartesian coordinates prior to tracking by the method proposed in [6]. Assume is the measurement in the twodimensional case where and
are the range and bearing measurement perturbed by independent Gaussian noise with a zero mean and standard deviation
and , respectively. Then, the converted measurement in Cartesian coordinates can be expressed as=  e^σ^2_θ/2 [rmcosθmrmsinθm]  (15) 
and its associated covariance matrix is
(16) 
where
(17)  
(18)  
(19)  
(20) 
IiiE Tracker estimate
Assume that each target belongs to one of known classes and the kinematic behavior of each target class is characterized by a linear dynamical model set with Markov property and corresponding transition matrix . The th dynamical model can then be represented by
(21) 
where is target kinematic state at time and is a white independent identically distributed (iid) Gaussian process noise with zero mean and covariance . Then the measurement equation can be written as
(22) 
where is assumed as iid Gaussian noise with zero mean and covariance given in (16).
Here it is assumed that there are number of IMM estimators [2] corresponding to target classes, respectively. These number of IMM estimators are set up for each target. The state estimate , associated covariance matrix , posterior probability of mode being correct given the measurement up to time , noted as , and the measurement likelihood function conditioned on mode , noted as , obtained by each IMM estimator and are provided to the subsequent recognition and fusion steps.
IiiF Recognition of tracker output
Given the measurement sequence up to time , the posterior probability of target being class , noted as , in the recursive Bayesian framework, is given by
where
(24)  
Also, is the posterior probability of mode being correct given the measurement up to time and is the likelihood function conditioned on mode of class at time . Finally, we have
and
(26) 
where is the prior probability that the target belongs to class .
IiiG Data association of recognized features
In any centralized or distributed tracking system, it is important to decide which of the received measurements are associated to the same target. As the reports may or may not contain the geo–location of the target, the data association algorithm should also be able to decide by considering only the features and probabilities that can be combined with each other. For instance, larger ships are associated with different types of emitter IDs and speed limits. This prior information must be available at any stage where association is necessary and processing must be applied to the received data to group them correctly with a common ID. The problem of association can also be addressed by the bearing and possibly range reports from ESM sensors.
IiiH Classification and identification
This step is the last processing block before the data is shown to the operator. If the data collected is in the form of probabilistic data, then the Bayesian framework is used for classification. However, if it is in the form of lists, for instance, belief functions are adopted to combine the data together. This can be done using Dempster’s rule of combination.
Iv Hierarchical design of modern fusion systems
Because there exist different information types extracted at different levels of data processing, it is important to design a hierarchical fusion system that can work with information from different steps in Figure 1. The revised architecture is shown in Figure 2.
According to Figure 2, other than the main flow of information (solid lines), there exist other possibilities to pass on the information at different stages. This is important when dealing with multiple ESM reports. In the next subsection, such a formulation is given for further development.
Iva Fusion of heterogeneous ESM information
To be able to explain the fusion process, a simple example is suggested here. Assuming a scenario with two ESM sensors, the problem is to fuse the information if they are not processed to the same level and also vary in terms of the features they carry. Note that if all the sensors send a set that includes speed information or its related features, then one can easily fuse them as they can map, with some approximations, to each other. The problem arises when they cannot be mapped into each other and yet it is desired to fuse all the information before classification. Such situations can happen when, for example, one ESM sends a report about speed and the other one send information about the threat category of the ship. These ESM datasets do not carry the same information, and they are processed at different levels. For an ESM sensor, the report can be of any of the form
(27) 
where different indexes show the possibility of different selected features that cannot be mapped to others directly. Assuming that there are ESM sensors in the surveillance region, possible combination can happen. If , then one possible combination is the set
(28) 
where the indexes outside the brackets show the sensor number. After associating data into different groups for different objects or targets, the new classifier must be able to handle the heterogeneous flow of information. Defining a classifier function as
(29) 
the final output would be a new set of features, possibly similar to the R&I declaration output to be sent to the operator.
One way to accommodate the fusion and classification of received data is to fill all the empty feature sets from collected data by different sensors if there exist non–overlapping data. In the case of overlapping, a simple fusion rule can be implemented as the data are of the same type and independent. After this processing, if there still exist some information gap to map all the data to the same feature set (R&I declaration), a new classification function must be used to deal with it properly.
V Simulation Results and Discussion
Now that the mathematical background and the system design of the sensor network have been defined, a simple example is discussed in this section to better understand the stpdf with R&I of ESM and kinematic data. Here a distributed R&I system is considered with two sensors, an ESM sensor and a radar. It is also assumed that a single ship is moving in the surveillance region as shown in Figure 3. To illustrate the impact of different processed data on the final classification output, it is assumed that the ESM report can be either the amplitude or the length. It is also assumed that the processed data from radar carries position and velocity in Cartesian coordinates.
The kinematics of the unknown target are sent to the fusion center as the kinematics vector and its associated covariance matrix . The amplitude measured by ESM is sent as and the length as , respectively. In the simulations, it is assumed that the kinematics are processed in the Bayesian framework and the noises are white and Gaussian, hence a covariance matrix contains the full information about the uncertainty of the reported kinematics. The amplitude has Rayleigh distribution [19]
and length has Gaussian distribution. The ESM reports, which are from different processing levels, are used for identification and classification separately and also together with the kinematics report to show the effect of adding ESM information for R&I.
Speed ()  

Class  
Class  
Class 
The classes used in the simulations and the parameters assigned to each feature are shown in Table III. Note that each scenario only uses one of the ESM features to fuse with the kinematic (velocity) feature. The initial speed of the unknown target is in Cartesian coordinates and the coarse is . The amplitude intensity is Rayleigh distributed with parameter and the length of the ship is . Also, the standard deviation of the uncertainty in the ship length’s measurement is . Taking in account all the information, it is expected that the data will be assigned to the third class.
The classification is done over Monte Carlo runs for a scenario of time stpdf of reporting values from ESM and radar. Here, two important measures are considered to compare performance of each data fusion process. The first is the average of class probabilities over time (Figures 4–9) and the second is the percentage of correctly reported class (Table IV).
Figures 4–6 show the class probabilities when only kinematic, length and amplitude features used separately for classification, respectively. In Figure 4, it can be seen that for the first few time stpdf, the class assignment is not stable. This shows that classifying based on only the kinematic of the unknown target can be misleading. Figure 5 shows a more consistent result in terms of classification, however, the results from first twenty time stpdf are not reliable. Finally, Figure 6 shows a more consistent result in terms of classification but with consistently lower probability assigned to the correct class. In order to classify the unknown target better, kinematic features can be coupled with either the data from the length of the ship (attribute feature), or the amplitude (signal level data from ESM). The results of these simulations are shown in Figures 7 and 8.
As shown in Figures 7 and 8, adding another feature to the classification can help the classifier distinguish between the unknown target features and other classes with higher accuracy. It is worth noting that amplitude and length information are from two different processing levels as stated in Figure 2. This shows that different ESM data, either attributes, signal level data or declared R&I data can be coupled with kinematics features for classification. In order to demonstrate what is the best performance that can be achieved for the classification of the unknown target, all three features are combined together to classify the target and the results are shown in Figure 9. Here, the two ESM features that are contributing are processed differently. This shows that it is possible to gain more information and to classify the targets with more accurately even if data are not homogeneous.
It is important to know how many of the class reports were correctly in the simulations. This measure can be shown by counting the number of correct reported class in each Monte Carlo run and then take the average from them in each case. The results of such calculation are shown in Table IV for all the cases considered in Figures 4–9.
Feature  Percent 

Velocity ()  
Amplitude ()  
Length ()  
As shown in Table IV, there is a significant advantage in using length or amplitude features in conjunction with the kinematic data. As the numbers suggest, using a single feature does not yield highly accurate classification in all the cases. It is worth noting that in two of the cases, i.e., and , the true class is assigned to the target in almost all the time stpdf.
The simulation shows that it is not necessary to consider only homogeneous ESM data, and different processing levels can be assumed for the system design. However, considering different processing levels should be done with some considerations for the type and uncertainty model of the data. One suggestion is to consider a combination of the Bayesian framework and belief functions to cover many of the situations that can occur in R&I using ESM data. To do so, a more general list of all possible signal level, attributes and R&I declared data must be available and suitable models should be assigned to them. Then, based on the suggested models, fusion and classification algorithms must be designed to address the problem in a consistent manner. This idea is in progress now.
Vi Conclusions
In this paper fusion of ESM and kinematic data was discussed along with proposal for a general architecture for classification and identification of ESM data. In order to comply with the requirements of R&I standards like STANAG , a hierarchical system design was proposed and further explored in the paper. The idea was to use different types of data from different processing levels and then combine them together to achieve better classification results. Simulation results showed that even adding one extra feature, from any level in the ESM processing chain, could improve the classification results solely based on kinematic.
Unless the fusion of heterogeneous ESM and kinematic data is considered in modern system design, the performance of the feature extraction and classification of modules will be limited. To achieve better performance, multisensor solutions must be considered using the architectures proposed in this paper that utilize non–homogeneous data. Processing, R&I and classification of non–homogeneous data from different sensors in realistic maritime solutions is in progress with consideration for the correlation among different levels of ESM data.
References

[1]
V. J. Aidala.
“Kalman filter behavior in bearings–only tracking applications”.
IEEE Transactions on Aerospace and Electronic Systems, vol. 15, no. 1, pp. 29–39, January 1979.  [2] Y. BarShalom, X. R. Li, and T. Kirubarajan. Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software. Wiley, NY, 2001.
 [3] Y. BarShalom, P. K. Willett, and X. Tian. Tracking and Data Fusion A Handbook of Algorithms. YBS Publications, Storr, CT, 2011.
 [4] J. M. Bernardo and A. F. Smith. Bayesian Theory. IOP Publishing, Bristol, United Kingdom, 2001.
 [5] I. Bilik, J. Tabrikian, and A. Cohen. “GMM–based target classification for ground surveillance Doppler radar”. IEEE Transactions on Aerospace and Electronic Systems, vol. 42, no.1, pp. 267–278, January 2006.
 [6] S. Bordonaro, P. Willett, and Y. BarShalom. “Decorrelated unbiased converted measurement Kalman filter”. IEEE Transactions on Aerospace and Electronic Systems, vol. 50, no. 2, pp. 1431–1444, April 2014.
 [7] R. R. Brooks, P. Ramanathan, and A. M. Sayeed. “Distributed target classification and tracking in sensor networks”. Proceedings of the IEEE, vo. 91, no. 8, pp. 1163–1171, August 2003.
 [8] S. Challa and G. W. Pulford. “Joint target tracking and classification using radar and ESM sensors”. IEEE Transactions on Aerospace and Electronic Systems, vol. 37, no. 3, pp. 1039–1055, July 2001.
 [9] C. Cheng, D. M. Lin, L. Liou, and J. B. Tsui. “Electronic warfare receiver with multiple FFT frame sizes”. IEEE Transactions on Aerospace and Electronic Systems, vol. 48, no. 4, pp. 3318–3330, October 2012.
 [10] N. Collins and C. Baird. “Terrain aided passive estimation”. National Aerospace and Electronics Conference, pp. 909–916, Dayton, USA, May 1989.
 [11] C. Davies and P. Hollands. “Automatic processing for ESM”. IEE Proceedings of Communications, Radar and Signal Processing, vol. 129, no. 3, pp. 164–171, June 1982.
 [12] H. Demers, G. Michaud, and D. Turgeon. “Identity multiassignment in ESM to radar fusion”. International Conference on Information Fusion, pp. 1–6, Florence, Italy, July 2006.
 [13] A. Gad and M. Farooq. “Data fusion architecture for maritime surveillance”. vol. 1, pp. 448–455, Annapolis, Maryland, USA, July 2002.
 [14] D. Gamma. EW101: A First Course in Electronic Warfare. Artech House, Norwood, USA, 2001.
 [15] D. L. Hall and J. Llinas. “An introduction to multisensor data fusion”. Proceedings of the IEEE, vol. 85, no. 1, pp. 6–23, January 1997.
 [16] A. Kawalec and R. Owczarek. “Specific emitter identification using intrapulse data”. First European Radar Conference, pp. 249–252, Amsterdam, Netherlands, Ocotber 2004.
 [17] X. Li, S. Wang, and J. Liang. “ESM–radar track correlation algorithms based on maximum likelihood estimation”. Electronics Optics & Control, vol. 1, pp. 012, 2007.
 [18] J. Matuszewski. “Knowledge–based signal processing for radar identification”. International Journal of Computing, vol. 7, no. 1, pp. 80–87, 2014.
 [19] A. Papoulis and S. U. Pillai. Probability, random variables, and stochastic processes. Tata McGrawHill Education, Columbus, OH, USA, 2002.
 [20] R. Poisel. Target Acquisition in Communication Electronic Warfare Systems. Artech House, Norwood, USA, 2004.
 [21] B. Ristic. “Target classification with imprecise likelihoods: Mahler’s approach”. IEEE Transactions on Aerospace and Electronic Systems, vol. 47, no. 2, pp. 1530–1534, April 2011.
 [22] B. Ristic and Ph. Smets. “Target classification approach based on the belief function theory”. IEEE Transactions on Aerospace and Electronic Systems, vol. 41, no. 2, pp. 574–583, April 2005.
 [23] B. Ristic and Ph. Smets. “Target identification using belief functions and implication rules”. IEEE Transactions on Aerospace and Electronic Systems, 41(3):1097–1103, vol. 41, no. 3, pp. 1097–1103, July 2005.
 [24] J. Roe, S. Cussons, and A. Feltham. “Knowledgebased signal processing for radar ESM systems”. IEE Proceedings of Communications, Radar and Signal Processing, vol. 137, no. 5, pp. 293–301, October 1990.
 [25] T. Smestad, H. Øhra, and A. Knapskog. “ESM–sensors for tactical information in air defence systems”. Systems Concepts for Integrated Air Defense of Multinational Mobile Crisis Reaction Forces, pp. 9, Valencia, Spain, May 2001.
 [26] NATO STANAG. “Standard: NATO  STANAG 4162: Identification Data Combining Process”. Technical characteristics of the NIS Identification Data Combining Process, 2001.
 [27] F. Voorbraak. “A computationally efficient approximation of Dempster–Shafer theory”. International Journal of Man–Machine Studies, vol. 30, no. 5, pp. 525–536, 1989.
 [28] C. K. Williams and D. Barber. “Bayesian classification with Gaussian processes”. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1342–1351, December 1998.
 [29] R. R. Yager. “On the Dempster–Shafer framework and new combination rules”. Information sciences, vol. 41, no. 2, pp. 93–137, 1987.
 [30] L. A. Zadeh. “Fuzzy sets as a basis for a theory of possibility”. Fuzzy sets and systems, vol. 1, no. 1, pp. 3–28, 1978.
 [31] L. A. Zadeh. “A simple view of the Dempster–Shafer theory of evidence and its implication for the rule of combination”. AI magazine, vol. 7, no. 2, pp. 85, 1986.
Comments
There are no comments yet.