Localization is an ubiquitous problem and has been researched across a wide range of fields. It has been studied for applications such as, underwater acoustics [underwatersonar], seismology [sidorovich1998two], sensor networks [safavi2017localization], and radar [AdaptiveRadar]. Extensive theory-based localization methods have been developed by signal processing researchers. These include model-based methods [alpay2000model], beamforming [BeamformingLocalization], subspace-based methods [zuo2018subspace], and multi-lateration [le2016closed]. In this work, we consider the application of localization in ultrasonic structural health monitoring (SHM).
In recent years, there has been an increased demand to maintain new and existing physical infrastructures, including buildings, aircraft, and oil rigs. This demand has increased interests in SHM. [ko2005technology]. SHM engineers create technology to detect, locate, and characterize damage in structures. In SHM, guided ultrasonic wave (GUW) systems are extensively studied for damage localization [rose2009successes, mitra2016guided]. GUWs have several attractive properties, such as a long scanning range, high penetrability, and greater damage sensitivity. These systems use ultrasonic sensors to transmit and / or receive waves across the structure. The sensors are often spatially dispersed to monitor large areas. Transmitters are excited at desired frequencies and the received signal is analyzed to locate the damage.
Damage localization is often solved as an inverse problem [friswell2007damage], where the damage is a point scatterer of incident waves. Methods such as MFP (an extension of the matched filter [giannakis1990signal]) is a popular model-based method for damage localization [harley2014data]. In MFP, a physical model for wave propagation in ideal conditions is adopted. The model (i.e., a multi-frequency waveform) is defined for every point in a spatial grid over which the damage is located. The model at every point is then compared with experimental data. The grid point with the maximum correlation is the estimated damage location.
As damage localization is a critical task in SHM [farrar2007introduction], analyzing sources of uncertainty is important. Many structures, such as aircrafts, experience extreme conditions. This alters the nature of the collected SHM data and necessitates robust signal processing to identify and locate damage in real-time prior to catastrophic failures and economic losses. As a result, there has been research on uncertainty quantification for SHM [lorenzoni2016uncertainty, sankararaman2011uncertainty].
Within the realm of source localization, researchers have studied the effect of uncertainties in sensor locations on the localization performance [EllipticLocalization, RecLocnError]. Yet, for commonly used damage localization techniques in SHM systems, such as model-based MFP, effects of environmental variations is a significant challenge [feuillade1989environmental], [StochasticMFP]. For example, wave velocity has a strong temperature dependence [Temperature, raghavan2008effects]. This leads to deviations from the physical model of wave propagation. The characterization of these external uncertainties is a challenging research problem [eybpoosh2014investigation], which limits the performance of model-based localization algorithms. Hence, it is of prime importance to account for these environmental uncertainties when building a SHM system [harley2012scale].
Data-driven methods, such as machine learning, are another approach for localization. Deep neural networks (DNN) have produced remarkable results on a variety of signal processing problems[InterferenceManagement, MIMODeepLearning, ResourceAlloc]. DNN based localization algorithms are found in various applications, such as underwater acoustics [niu2017source], sound source localization [yalta2017sound], and object localization [tompson2015efficient]. Neural networks learn non-linear representations from input to output [bengio2012deep]. This makes them ideal for SHM localization, which involve complex dependencies between inputs and outputs [harley2017managing]. Yet end-to-end learning of desired parameters, as is common in data-driven paradigms, is not well-suited for parameter estimation problems where uncertainty has to be accounted for suitably. This motivates us to incorporate and represent uncertainty in damage localization.
We propose a DNN based framework for damage localization in presence of uncertainty. We model environmental uncertainty as randomness in the wave velocity / wavenumber. We model sensor noise as additive white Gaussian noise. We model the damage location as a multi-modal probability distribution. The distributional parameters are obtained as outputs from the DNN. The variance estimate of the output distribution represents predictive uncertainty. By using multiple mixture components in the distribution, this framework can identify multiple damage locations.
We train and validate the DNN model using simulated data. The DNN framework is compared with MFP for varying levels of uncertainty. The performance of the two frameworks is also compared with varying numbers of damages. The proposed framework demonstrates robust localization that quantifies predictive uncertainty in the process. Our approach demonstrates a localization error of m as compared to m with MFP in presence of environmental uncertainty and dB noise. In addition, we show that the predictive uncertainty scales as environmental uncertainty increases to provide a statistically meaningful metric for assessing localization accuracy.
Uncertainty aware damage localization
Ii-a Localization Input Uncertainties
Lamb waves are a specific type of guided waves found in isotropic plates and used for damage localization [alleyne1992interaction]. The Lamb wave frequency response is defined by
is the frequency domain representation of the signal and is modeled as the summation acrosswave modes. The function is the frequency and mode dependant wavenumber (known as the dispersion relation) and is the distance travelled by the wave. Fig. 1 illustrates the impulse response of (1) with modes (the zeroth symmetric mode and zeroth asymmetric mode).
In physical / engineering systems, researchers have formalized uncertainty into two types: aleatoric and epistemic [chowdhary_dupuis_2013]. Aleatoric uncertainty varies with each experiment. In the damage localization scenario, random sensor noise is the aleatoric uncertainty. We represent random sensor noise as additive white Gaussian noise. Epistemic uncertainty is the systematic uncertainty or lack of underlying knowledge in an experiment. Uncertainty because of external/environmental factors, such as temperature, represents epistemic uncertainty. As we saw earlier, variations in temperature (as well as other environmental factors) affect wave velocity [raghavan2008effects]. The dispersion relation for a wave relates group velocity to the frequency-dependent wavenumber according to [weaver1981dispersion]
Fig. 1(a) and 1(b) illustrates how uncertainty negatively affects MFP. The maximum value of the heat-maps correspond to the predicted damage location. As uncertainty increases, MFP is less accurate and less localized. The maximum value also reduces with uncertainty. Researchers have proposed data-driven methods for localization with uncertainty [harley2014data], but these techniques can require an impractical amount of calibration data and do not provide predictive uncertainty. As discussed previously, DNN based frameworks can leverage the data available from observations and learn robust representations in a localization setup. This motivates us to build an uncertainty-aware DNN framework for damage localization.
Ii-B Challenges with DNN-based Localization
Consider the following localization setup. Let be the input wave data from an array of sensors and let be the damage location (ground truth). Let , be a non-linear function from input to output, learnt by a DNN during training. Let denote the DNN prediction for the damage location
For regression tasks (localization is a regression task), the traditional loss function is the mean squared error (MSE) between prediction of the DNN and the ground truth
Note that MFP can also be expressed as an approach which minimizes the MSE between the model and data [harley2014data].
This DNN based localization setup has a number of challenges. First, the mapping learned is usually an arbitrary function with no statistical information. By training such a network with an objective function of squared error, we obtain the approximate conditional mean and variance of the prediction through Monte-Carlo simulations [bishop1994mixture]. Yet these two statistics are not enough to uniquely identify the output probability distribution. To represent uncertainty at the output, an appropriate probability distribution must be inferred.
Second, the output of an optimization objective in (4) is a point estimate. A critical drawback with a model that generates point estimates is the scenario when we have a one-to-many mapping from input to output. Such a situation is shown in Fig. 3
. The neural network approximately identifies the mean of the true damage locations. This is a typical localization result of a standard feed-forward DNN (changing the DNN hyperparameters does not significantly change the result) when multiple damage locations are present. Hence, a standard DNN is not appropriate for multi-damage localization tasks.
Ii-C Addressing Challenges using Mixture Density Networks
To address these deficiencies in traditional neural networks, we first add a source of variability at the input stage. The predictive uncertainty (uncertainty that propagates from input data to the output) is then represented by mixture density networks, proposed in [bishop1994mixture]. A mixture density network (MDN) models the conditional output probability distribution as a specific class of mixture model. In an MDN, instead of optimizing the MSE as in (4), the conditional output likelihood is maximized. This makes MDN an attractive option to infer relevant probability distribution parameters. Second-order statistics, such as variance, can represent predictive uncertainty.
There has been renewed interest in MDN’s in the context of representing uncertainty in systems [choi2018uncertainty] and for applications that involve multi-modal outputs [zen2014deep, wang2016gating, wang2017autoregressive]
. With a MDN, we obtain two complementary advantages. First, we have the flexibility of learning non-linear representations from input to output by virtue of a deep learning based model. Second, we also have statistical information about the output distribution by virtue of likelihood optimization. In contrast, many uncertainty estimation techniques in deep learning take a Bayesian approach[bernardo2009bayesian, neal2012bayesian, gal2016dropout]. These techniques typically choose a prior distribution and develop a scheme for approximating the posterior distribution. The performance is often limited by computational constraints. With an MDN, the parametric estimation is free from the computational complexities.
There exist a number of multi-modal distributions that mixture density networks can use [mclachlan2004finite]. We choose a GMM since a GMM with the appropriate number of components is capable of representing an arbitrary function within a finite error threshold [kostantinos2000gaussian]. Formally, we model the probability density of our output conditioned on the input as a multi-variate GMM with components, given by
with the Gaussian mixture component given by , and mixing coefficient . The mixing coefficients must satisfy
In an MDN, we have the traditional architecture of a neural network with hidden layers. The last layer (referred to as the mixture density network layer) outputs a vector of length, where is the number of mixture components and is the dimension of output space (i.e., 2 in our case). This layer outputs elements corresponding to the parameters of the mixture distribution ( elements corresponding to the mean estimates for all mixture components, elements corresponding to variance estimates for all mixture components and elements corresponding to mixture probabilities for all mixture components). Let us denote the output vector by as follows
Since the only restriction on the component means is that they have to be real-valued, the relation is as given in (8). Since the component variances have to be positive, we choose a monotonically increasing function (exponential) and we have the relation as in (9). For the mixing coefficients to satisfy the condition in (6), we use a soft-max function in (10). Together, the component means, variances, and mixing probabilities specify a unique GMM. An illustration of an MDN with two mixture components at the output is shown in Fig 4.
In an MDN, the conditional output likelihood as given in (5) is maximized. The distribution parameters to be used for the likelihood calculation are obtained from the preceding three equations. Typically, optimization in the deep learning paradigm is a batched process. The batch-data likelihood given by the joint conditional distribution of all output samples in a batch has to be calculated in the optimization process. Consider we have
samples in one batch. Assuming independent and identically distributed data (i.i.d), the joint distribution can be written out as the product of individual output probabilities
For computational ease, the log-likelihood is optimized instead of likelihood, given by
The distributional parameters of the GMM can then be learned by maximizing . Note that incorporating the likelihood as a loss function is non-trivial and there are computational issues that have to be considered, such as unstable learning and collapsing modes. We refer to [mdn] for practical implementations of an MDN.
Ii-D Damage Localization Mixture Density Network
Due to its capabilities, we use an MDN trained on wave data to obtain a multi-modal distribution to estimate damage location(s) and represent predictive uncertainty. We denote our localization neural network as UR-Net (short for uncertainty representation network), shown in Fig 5. As illustrated in Fig. 4, UR-Net is a deep feedforward network with mixture density network layer as the final layer.
The input to UR-Net is of size and represents measured pairs of time-domain signals. We have sensor transmitter-receiver pair measurements (with sensors sparsely distributed across the space) and discrete times points. We train the neural network with instances of our guided wave model in (1). We incorporate frequency-dependent wavenumber uncertainty into the simulation. We also add Gaussian noise to model sensor noise in the simulation. This is shown in the left-hand-side of Fig. 5.
This paper considers active localization, where a signal is sent from a transmitter, interacts with the damage, and is then measured by the receiver [quazi1981overview]. In active localization, baseline subtraction is often first applied, removing the direct arrival and other reflections from the structure. Due to environmental variations, baseline subtraction is often imperfect [dawson2015challenges]
, but we do not consider these imperfections in this paper. We then standardize the data (zero mean, unit standard deviation) before inputting it into the network.
As discussed in previous sub-section, the output of UR-Net (for d-dimensional localization) is of size , where is the number of mixture components (corresponding to the number of potential damage locations) and is the output space dimension ( = 2 in our case). Note that can be larger than the true number of damage locations. Unlikely component locations will have small likelihoods. At inference time, UR-Net outputs the parameters conditioned on that one sample of test wave data. Predictive uncertainty at the output is represented by the estimated mixture component variance(s). The outputs can be visualized as a contour plot, shown at the right-hand-side of Fig. 5. The contour is the multi-modal distribution for the output conditioned on a particular input. The width of each Gaussian represents its predictive uncertainty.
Ii-E Matched Field Processing Comparison
We compare UR-Net with matched field processing (MFP), a model-driven localization technique commonly used in underwater acoustics [gingras1997electromagnetic], seismology [sidorovich1998two], robotics [argentieri2007broadband], and radar [yardibi2010source]. For MFP, there exists a query grid of dimensions . This grid would be superimposed on a structural plate or panel (such as on an aircraft). We assume the plate has point damage at some location, modeled as a point scatterer.
Let denote the signal for frequency and sensor pair . Similarly, let be our mathematical model for damage at point in the grid. We define with the Lamb wave model described in (1). The correlation between the mathematical model and experimental data at point is denoted by
where denotes complex conjugate. The numerator of (13) denotes the correlation between the experimental data and the mathematical model with the damage location assumed at a specific point in the grid. The denominator is a normalization factor. The location estimate by MFP is given by
where corresponds to some coordinate location.
Fig. 6 compares the MFP output with UR-Net output. MFP obtains a spread out and granular contour, representing the correlation between the physical wave model and experimental data. The contour obtained from UR-Net is a visualization of a mixture model, where large values correspond to locations with a high likelihood. Hence, uncertainty with UR-Net is encapsulated by the distributional parameters. With MFP, the uncertainty is defined by similarities between the data and the mathematical model.
Iii Simulation Setup
Iii-a Data Setup
We use Lamb wave simulations in (1) to generate both training and testing data. UR-Net and MFP are applied to the same dataset. We consider a region of a m by m plate. Damage(s) is modeled at random point(s), as shown in Fig. 7. While there are existing methods for optimal sensor geometry [OptSensorPlace], [OptimalGeometry], we use random sensor placement to unbias the results based on sensor configuration. We simulate acoustic data using equation (1) with sensors ( unique sensor pairs) placed at random locations on the plate. We use equally spaced frequencies from to kHz to simulate the waves. For UR-Net, we convert the input signal into the time-domain.
The uncertainty due to environmental factors is modeled by introducing a random multiplicative effect on the wavenumbers. The modified wavenumbers are defined as . We statistically model as
represents a truncated Gaussian distribution with mean, variance , and truncated between . We vary the uncertainty in wavenumber by varying (e.g., = 0.15 implies that is sampled from a distribution truncated between 0.85 and 1.15). We use a truncated Gaussian to simulate a wavenumber distortion that is as close to a realistic environmental scenario as possible.
Iii-B UR-Net implementation
UR-Net has a total of hidden layers, of which, are fully connected hidden layers. First hidden layer has nodes, second hidden layer has nodes, and third hidden layer has nodes. The output layer is the MDN layer. The number of hidden layers and nodes in each layer is chosen after manual experimentation to maximize performance. This choice of hyperparameters gives the fastest convergence in terms of number of iterations.
We simulate Monte Carlo samples (i.e., collections of measurements with different SNRs and values) for training UR-Net. We use the Adam optimization algorithm [kingma2014adam] for training the network. We also regularize UR-Net by using dropout [srivastava2014dropout], which is a common practice. Dropout randomly drops out nodes from the neural network during training time with a pre-specified probability. This potentially leads the network to learn weights in a robust manner by preventing overfitting.
We use 3-fold cross-validation in our model training process. We observe that for the training phase, dropout probability has to be increased between to
as the input uncertainty increases to avoid overfitting. To choose the right value of dropout probability, we monitor the training and validation loss trend. The framework is implemented using Keras framework[chollet2015keras]. We report results and compare the performance of UR-Net and MFP on a separate test set consisting of samples. The test data and training data are configured with the same noise level, amount of wavenumber distortion (), and number of damage locations.
Iii-C MFP Setup
As our simulation consists of a unit plate, we set unit grid dimensions for MFP (i.e L, W = 1). We define equally spaced grid points on each axis (giving a total of 10000 grid points). For ease of localization with MFP, we divide the query grid into 4 equal sub-grids. We then simulate damage(s) in distinct sub-grids. The location(s) in distinct sub-grids (that have a damage) with the maximum correlation value are chosen as the MFP estimate(s). For comparison, we use the same test data as used for UR-Net.
Iv Results and Discussion
During inference time, every predicted component mean for UR-Net is assigned to the closest true damage location. The performance of UR-Net is quantified on the test set using an average localization error (ALE) metric, defined as:
Here is the total number of samples in the data-set and is the number of damages in the sample. Further, , are the actual and predicted damage locations respectively for the damage in the sample. ALE is the Euclidean distance between the actual and predicted damage location, averaged across each sample of the dataset. A smaller value of ALE indicates that the localization prediction is more accurate. For all of the further experiments, we choose a GMM with 3 components unless mentioned otherwise. For MFP, the ALE is computed by plugging in the estimated damage location(s) in (16).
Iv-a Performance comparison at varying levels of noise
Fig. 8 shows the ALE comparison of UR-Net and MFP at different sensor noise levels for a simulation setup with a maximum of 2 damages. In this illustration, the error bars represent the standard deviation in the prediction errors. In this simulation, wavenumber distortion (
) has been set to 0.15. Sensor noise is modeled as additive, white Gaussian noise (AWGN). Here, signal-to-noise ratio (SNR) is defined in as
where is the power of the baseline-subtracted signal.
We observe that UR-Net has a substantially better performance than MFP. We also observe a trend of decreasing ALE as the signal-to-noise ratio (SNR) goes up. UR-Net has an error of 0.0625 m at 25 dB as compared to 0.1425 m for MFP. While the error at -50 dB is 0.43 m and 0.54 m, respectively, for the two approaches. Note that the error convergences to approximately
m, one-half the length and width of plate. Based on a Monte Carlo analysis, the expected error of randomly sampling true and predicted locations with a uniform distribution would be approximatelym.
Iv-B Performance comparison at varying levels of uncertainty
Fig. 9 illustrates the performance of UR-Net and MFP at varying levels of uncertainty in the simulation setup. The leftmost bar represents data with 5 dB SNR. The next three bars represent scenarios with increasing levels of wavenumber uncertainty and 5 dB SNR. In the case with only noise, MFP has optimal performance ( m), while performance of UR-Net is comparably inferior ( m). This is reasonable since MFP is derived from the matched filter, an optimal detector in white Gaussian noise.
The next three bars represent cases where we have wavenumber distortion and dB SNR (both epistemic and aleatoric uncertainty). We observe that while the performance of UR-Net sees only a small decrease with varying uncertainty levels, the performance of MFP decreases consistently with increasing uncertainty level. MFP cannot handle uncertainty in wavenumber since the underlying model of the MFP is fixed. In contrast, since our approach learns from the available data to infer the conditional output distribution, it is more robust to uncertainty.
Fig. 10 compares the probability of true damage being within a confidence interval of a component mean for different values of wavenumber distortion (). A high probability of damage being in the confidence interval tells us that the mean and variance parameters of the inferred conditional distribution are able to reliably model the uncertainty in the conditional distribution of the damage location given the input data.
We observe that the curves for all values of are closely overlapping in the SNR range of to dB.This trend implies that even with increasing levels of uncertainty in wavenumber, UR-Net is able to infer appropriate parameters for the conditional output distribution. With the evidence of a trend in this figure and the trend of nearly constant performance of UR-Net in the previous figure (Fig. 9), we observe that the framework performance is robust to uncertainty in velocity (i.e., ).
Iv-C Representing predictive uncertainty with UR-net
While robustness to uncertainty is one goal, another goal is to represent the predictive uncertainty. We explore the use of component variance parameter output by UR-Net in (9) to represent predictive uncertainty. Fig. 11 illustrates the maximum value of component variance (worst value) obtained from UR-Net during inference time with varying levels of uncertainty. We observe a trend of increasing predictive variance as uncertainty () increases. The leftmost bar representing a noise-only scenario has the lowest predictive variance () while the rightmost bar with dB SNR and has the maximum predictive variance (). The component variance parameter outputted by the UR-Net thus can act as an efficient way to represent uncertainty that propagates from input to output.
Fig. 12 illustrates the average log-likelihood for the validation data set as outputted by the UR-Net. This comparison is in the presence of varying levels of uncertainty. We observe a trend of decreasing likelihood with increasing level of uncertainty. With only noise, the log-likelihood is 3.5 and drops to 3.0 when we have both noise and . Here, likelihood can be thought of as a measure of “goodness of fit” for the inferred model.
Iv-D Performance comparison with varying number of damages
Fig. 13 shows the comparison between UR-Net and MFP with a varying number of damages in the simulation setup. In the simulation setup, we have wavenumber distortion () and dB sensor noise. For UR-Net, we use one more mixture component than number of damages in the simulation setup.
The performance of UR-Net is superior to MFP for all number of damages investigated here. The localization error (ALE) for UR-Net is m and m for 1 and 4 damages respectively. Similarly, the corresponding error values for MFP are m and m respectively. While the performance of both approaches drops down with an increasing number of damages, MFP performance decreases at a faster rate as compared to UR-Net.
We proposed an uncertainty-aware DNN based framework, UR-Net, for damage localization with guided waves. The motivation for our framework was the need to assess and represent the uncertainty due to environmental variations. Our framework outputs parameters of an appropriately chosen mixture model. The outputted mixture component means are estimates for the damage locations and the component variances are a measure of the predictive uncertainty.
We showed the results of various simulation runs that compare the localization performance of UR-Net with MFP. Our framework shows better performance than MFP in presence of aleatoric and epistemic uncertainty. We also analyze the effect of increasing levels of uncertainty on the performance of our approach. Finally, we studied the effect of increasing the number of damages in the performance of both MFP and UR-Net.
We showed that UR-Net has two advantages over MFP: robustness to uncertainty due to environmental variations and being able to represent damage location as a probability distribution for a multi-damage scenario. Representing damage location as a probability distribution offers us a principled way of representing uncertainty through confidence intervals and / or variance.
In the future, the approach can be extended for random structure geometries. Further work will also include analysis of the resolution achievable by this framework. This can serve as a metric of comparison with traditional localization algorithms. In the future, there is also a potential to explore ways for estimating damage intensity using this framework.
This research is supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0126 and by the National Science Foundation under award number EECS-1839704.