Spectral Variability Augmented Sparse Unmixing of Hyperspectral Images

10/19/2021
by   Ge Zhang, et al.
0

Spectral unmixing (SU) expresses the mixed pixels existed in hyperspectral images as the product of endmember and abundance, which has been widely used in hyperspectral imagery analysis. However, the influence of light, acquisition conditions and the inherent properties of materials, results in that the identified endmembers can vary spectrally within a given image (construed as spectral variability). To address this issue, recent methods usually use a priori obtained spectral library to represent multiple characteristic spectra of the same object, but few of them extracted the spectral variability explicitly. In this paper, a spectral variability augmented sparse unmixing model (SVASU) is proposed, in which the spectral variability is extracted for the first time. The variable spectra are divided into two parts of intrinsic spectrum and spectral variability for spectral reconstruction, and modeled synchronously in the SU model adding the regular terms restricting the sparsity of abundance and the generalization of the variability coefficient. It is noted that the spectral variability library and the intrinsic spectral library are all constructed from the In-situ observed image. Experimental results over both synthetic and real-world data sets demonstrate that the augmented decomposition by spectral variability significantly improves the unmixing performance than the decomposition only by spectral library, as well as compared to state-of-the-art algorithms.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 9

page 10

04/30/2018

Hyperspectral unmixing with spectral variability using adaptive bundles and double sparsity

Spectral variability is one of the major issue when conducting hyperspec...
01/03/2017

Semi-Supervised Endmember Identification In Nonlinear Spectral Mixtures Via Semantic Representation

This paper proposes a new hyperspectral unmixing method for nonlinearly ...
07/23/2020

Illumination invariant hyperspectral image unmixing based on a digital surface model

Although many spectral unmixing models have been developed to address sp...
09/20/2019

Deep Generative Models for Library Augmentation in Multiple Endmember Spectral Mixture Analysis

Multiple Endmember Spectral Mixture Analysis (MESMA) is one of the leadi...
01/22/2015

Estimating the Intrinsic Dimension of Hyperspectral Images Using an Eigen-Gap Approach

Linear mixture models are commonly used to represent hyperspectral datac...
02/14/2019

Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing

Endmember (EM) spectral variability can greatly impact the performance o...
10/29/2018

An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing

Hyperspectral imagery collected from airborne or satellite sources inevi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

With the rapid development of remote sensing technology, hyperspectral image (HSI) collecting abundant spectral information with hundreds of contiguous bands, has been widely used in different kinds of applications, such as geoinformation science, space research, material mapping, and geological exploration [1]. However, the phenomenon of mixed pixels is widely existed due to insufficient spatial resolution of imaging system, changeable atmospheric environment, and complex distribution of ground objects. The decomposition of a mixed pixel into spectral signatures (endmembers) with corresponding proportions (abundances) is known as hyperspectral unmixing [2].

Linear mixing model (LMM) has been widely used for spectral mixture analysis as its computational tractability and flexibility and it supposes that the spectra collected by the imaging spectrometer are a linear combination of endmembers weighted by their corresponding abundance fractions [3]. However, nonlinearity and spectral variability affect the performance of spectral unmixing [4]

. Techniques based on geometry, statistics, and nonnegative matrix factorization extract endmembers directly from the HSI. In general, an endmember extraction step is applied, and then, abundance value for each pixel is estimated. There are many algorithms for endmember extraction, such as N-FINDR

[5], pixel purity index (PPI) [6], and vertex component analysis (VCA) [7]. Nevertheless, these algorithms require pure pixel assumption, and it is not always satisfied due to the spatial resolution. Others of them extract virtual endmembers without physical meaning in a given HSI [8].

Recently, due to the wide availability of spectral libraries, like the U.S. Geological Survey (USGS) digital spectral library, a semi-supervised spectral unmixing approach, called sparse regression-based method, has been shown to circumvent the drawbacks introduced by such virtual endmembers and the unavailability of pure pixels [9, 10]

. It assumes that the mixed pixels can be expressed in the form of linear combinations of a small number of pure spectral signatures from a (potentially very large) spectral library known in advance. Another significant advantage is that it does not need to extract endmembers from the hyperspectral data or estimate the number of the endmembers. Generally, the number of endmembers in the scene is small compared with the number of endmembers in the spectral library, which means that only a small number of endmembers contribute to the mixed pixel. Therefore, the abundance vector of the mixed pixel estimated by sparse unmixing is expected to be sparse. These new perspectives introduced by sparse unmixing fostered advanced developments in the field

[11, 12, 13].

In the past few years, several algorithms have been developed to enforce the sparsity on the solution of SU [14, 15]. The sparse unmixing algorithm via variable splitting and augmented Lagrangian (SUnSAL) [9] adopts the regularizer on the abundance matrix to measure the sparsity of the abundance vector in each pixel. The collaborative SUnSAL (CLSUnSAL) [16] introduces the regularizer to constrain the pixels in HSIs to share the same active set of endmembers and ensure that all the abundance vectors exhibit global row sparsity. Based on the alternating direction method of multipliers (ADMM) [17], the corresponding sparse regression problems are all convex and can be settled efficiently. Moreover, the spatial information acts a pivotal part in sparse unmixing as the corresponding problem is usually made easier on a local scale [18]. Total variation (TV) [19, 20, 21] regularization exploits the spatial information via a first-order pixel neighborhood system to significantly improve the unmixing results, but it may yield oversmoothness and blurred boundaries. Zhang et al. [22] proposed a spectral Cspatial weighted sparse unmixing (SWSU) framework constrained simultaneously from the spectral and spatial domains to further imposing sparsity on the solution. To obtain spatial-contextual information, a fast multiscale spatial regularization unmixing algorithm (MUA) was proposed by Borsoi et al. [23]. In this algorithm, superpixel-based segmentation is performed before unmixing to form a coarse domain, and the unmixing result for the coarse domain forms a multiscale spatial regularization for unmixing of the original domain. Another solution is to focus on the local spectral similarity of the HSIs rather than the simple proximity of the location. The local spectral similarity preserving (LSSP) constraint [24]

was proposed to preserve spectral similarity in a local region given that adjacent pixels share not only the same endmembers with high probability but also approximated fractional abundances. Meanwhile, the low-rank constraint of the abundance matrix has been increasingly adopted for sparse unmixing, providing a new perspective for spatial correlation

[25]. Giampouras et al. [26] simultaneously imposed single sparsity and low rankness on abundance matrices, taking into account both sparsity and spatial correlation information in HSIs. The joint-sparse-blocks and low-rank unmixing (JSpBLRU) algorithm [27] further imposed the joint-sparsity-blocks structure and low rankness on abundance matrices for pixels in a sliding window. Recently, the sparse unmixing method named superpixel-based reweighted low-rank and total variation (SUSRLR-TV) [28] was proposed by using superpixel segmentation to extract the homogeneous regions, and imposing a low-rank constraint to promote the correlation of each superpixel s abundance matrix.

However, the spectral signatures of the materials contained in hyperspectral images can be significantly affected by spectral variability, which results from different imaging conditions including atmospheric effects, illumination, topographic changes, and the intrinsic variation of the spectral signatures of the materials (i.e., due to physicochemical differences), especially in a hyperspectral image with a higher spatial resolution [29, 30, 31]. Therefore, the LMM hardly makes an accurate unmixing in reality, due to such ubiquitous error that passively transfer the unpredicted errors into LMM [32]. The existing methods could be basically divided into two groups: 1) dictionary or bundle-based approaches [33, 34], which try to model endmembers by a certain number of instances of each material, 2) model-based approaches, which describes the endmember variability by a specific statistical distribution [35, 36, 37] or by incorporating the variability in the mixing model based on physically motivated concepts [38, 39, 40]. This work is developed following the first approach.

The key idea in dictionary or bundle-based approaches consists in extracting several instances of each endmember in order to build a dictionary, which is then used for spectral unmixing, offering more than one representative spectral signature per material [41, 42]. Representing each endmember class with large spectral libraries of spectra as a priori

does not require the assumption of any particular spectral distribution within a single endmember class. Methods for this category select the appropriate spectra, or bands, from different spectral sets according to pre-defined criteria for weakening the effect of endmember variab ility, which can be roughly divided into four groups: multiple endmember spectral mixture analysis (MESMA), sparse unmixing, spectral transformations, and machine learning.

The basic principle behind MESMA [43] is to iteratively search for the combination of endmember signatures in the library that, among all possibilities, enables the closest reconstruction of each observed pixel under the LMM. The MESMA algorithm and its variants formulate spectral unmixing as a computationally demanding optimization problem and achieve good quality [44]. Chen et al. [45] proposed a sparse multiple-endmember spectral mixture model (SMESMM) by using a block sparse algorithm to obtain an initial block sparse solution and resolving the mixed pixel using the selected land cover materials. Zhang et al. [46] incorporated spatial information in MESMA by using segmentation algorithms to divide the image into different homogeneous objects, which are then unmixed individually by using a library that is also constructed from object based spectra. Borsoi et al. [47] leveraged the power of deep generative models to learn the statistical distribution of the endmembers available in the existing libraries, and then drew new samples to augment the spectral libraries, improving the overall quality of the unmixing process.

Compared to MESMA, sparse unmixing formulations use mathematical relaxations that are computationally easier to solve. A typical approach [48] was to modify the LMM for unmixing mineral spectra in mining applications by including an additional term representing the mixture of the background spectrum of the endmembers. This background spectrum was defined as the low-frequency part of the spectral signatures and estimated a priori from the library as a parametric function of smooth splines. The performance of an norm-based sparse unmixing framework under this model was reported to be similar to MESMA, albeit at a much smaller computational cost.

In the spectral transformations group, instead of the original reflectance data, transformed spectral information with less spectral variability are used as input to the SMA, like derivative spectra and wavelet transformed spectra [49]. However, spectral transformations are empirically oriented techniques and require a significant degree of expert knowledge about the underlying application [50]. Other types of methods are more machine learning-oriented, which allow to learn the function linking the endmembers available in the dictionary (or in a supervised fashion using an a priori available dictionary). The training can be performed for instance by simulating mixed pixels with endmembers from the dictionary in different proportions [51], or models the latent function between spectra and abundances in a training set and predicts abundances from a given pixel spectrum under Gaussian process framework [52]. Machine learning algorithms provide more flexible ways but these methods have the drawbacks of not explicitly modeling the variability, and at a large computational complexity.

Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, which propagates significant modeling errors throughout the whole unmixing process and compromises the quality of the results. The recent library-based SU methods address spectral variability by using libraries of spectra that had to be acquired a priori (e.g., through in situ measurements) without introducing explicit spectral variability term, which limited the applicability of these approaches. Therefore, a spectral variability augmented sparse unmixing model (SVASU) is proposed, which provides a new way to construct spectral variability explicitly from the spectral library for sparse unmixing. Finally, extensive experiments over both synthetic and real-world datasets are conducted to validate the effectiveness and robustness of the proposed SVASU method for hyperspectral sparse unmixing.

The remainder of this paper is organized as follows. Section II introduces the basic idea of sparse unmixing. Section III presents the proposed SVASU model for SU of hyperspectral images. Section IV reports and discusses experimental results on both synthetic and real datasets. Finally, Section V draws a conclusion.

Ii Sparse Unmixing

Fig. 1: Basic flowchart of sparse unmixing through SVASU.

Ii-a Linear Mixture Model

The Linear Mixture Model (LMM), which is well-known for its simplicity and explicit physical meaning, has been widely utilized to model the relationship between mixtures and endmembers. In the LMM, the photons reflected from different ground objects within one pixel are assumed not to interfere with each other. As a result, the observed hyperspectral image matrix is assumed to be the product of endmember matrix and its corresponding abundance matrix plus a noise matrix . Their relationship can be formulated as:

(1)

in which is the number of bands, is the number of endmembers, and is the number of pixels in the image. Here, the -th column vectors of and , denoted by and , correspond to the -th endmember and the abundance of the -th spectral pixel , respectively.

In order to make the LMM physically meaningful, the abundance sum-to-one constraint (ASC) and the abundance nonnegative constraint (ANC) must be imposed, which can be expressed as:

(2)
(3)

Generally, the ASC can be embedded into this model by adding an additional pseudo band to the data matrix and the endmember matrix [53]. Therefore, in the following analysis, only the ANC is explicitly considered.

Ii-B Sparse Unmixing

Due to the wide availability of spectral libraries, sparse unmixing models use mathematical relaxations that are computationally easier to solve. Let be a large spectral library containing spectral signatures, and denotes the abundance maps corresponding to library for the observed data . As the number of endmembers involved in a mixed pixel is usually very small when compared with the size of the spectral library, the abundance matrix contains many zero values, which means is sparse. Thus, the unmixing problem can be formulated as an optimization problem,

(4)

where is the Frobenius norm, is the -norm which denotes the number of nonzero components of a vector, and is a regularization parameter used to adjust the weight of the sparsity in this model.

Problem (4) is nonconvex and difficult to be solved. However, the -norm constrained sparsity can be relaxed using norm, by which the sparse unmixing algorithm via variable splitting and augmented Lagrangian (SUnSAL) [9] using the norm to solve the unmixing problem as follows,

(5)

where with being the -th column of . However, the Problem (5) is heavily influenced by the high correlation of spectral libraries due to the underdetermined nature. To admit a sufficiently sparse solution for sparse unmixing will guarantee a more accurate abundance estimation. In [16], the Collaborative Sparse Unmixing via variable Splitting and Augmented Lagrangian (CLSUnSAL) algorithm imposes the joint sparsity with an mixed norm among the endmembers simultaneously for all of the pixels, whose objective function can be defined as follows,

(6)

where denotes the -th line of and is the so-called mixed norm.

Iii Sparse Unmixing Via SVASU

A considerable number of such sparse unmixing methods has been developed using libraries of spectra that originally had to be acquired a priori (e.g., through in situ measurements), which used to limit the applicability of these approaches. An important recent development concerns methods that can extract spectral libraries directly from the observed images or generate them using physics-based mathematical models of material spectra. This supports the widespread applicability of library-based sparse unmixing techniques in situations where spectral libraries are not available or cannot be built.

However, the spectral signature variability affects the performance of sparse unmixing task since that the inaccurate endmember representation undoubtedly results in the propagation of errors in abundance estimates derived using SMA. Thus, to address spectral variability problem, the spectral variability library is introduced to assist the spectral libraries for better signature approximation in sparse unmixing. Specifically, the introduction of spectral variability library makes a spectrum signature divided into two independent representation parts, which allows the inherent spectral variability into materials to be represented separately and explicitly.

Iii-a Extraction of In-situ Spectral Library by SPEE

In a hyperspectral image, pixels from a homogeneous ground object often present in adjacent areas, which result in lots of pure spatial neighborhoods in the image. If all these pure spatial neighborhoods can be detected, their representative spectral signatures can be selected as endmember candidates. Different endmember candidates can be considered as coming from homogeneous ground objects provided that the spatial neighborhoods they represent cover pixels in common [54]. Based on this spatial refinement scheme, the remaining spatially independent endmember candidates become more representative since they represent larger areas. The number of endmember candidates to constrcut the in-situ spectral library will also be significantly reduced. Note that the in-situ spectral library is extracted from the observed image (refer to [54]), containing multiple spectra of each possible material (a total of materials), written as

(7)

in which represents the library subset corresponding to the -th possible material and is composed of spectral signatures of the current material.

Iii-B Segmentation of Endmember Library and Spectral Variability Library

Through the previous section, we have obtained the image spectral library including the spatially independent and spectrally pure pixels in the homogenous region according to the pixel purity. Principal Component Analysis (PCA) is a classic method widely used in signal feature extraction, data dimensionality reduction and compression. It retains the main information of the original image data by retaining the first

principal components corresponding to the larger feature value, and the remaining principal components correspond to noise, spectral variability, and/or outliers.

In this section, PCA is performed on its covariance matrix after standardizing the pixel data of the in-situ spectral library as

(8)

where

are the eigenvalues of its covariance matrix

, and

is composed of eigenvectors of the corresponding eigenvalue matrix

. Among all the principal components, the first correspond to the unique feature types of various pure pixels in the homogeneous area, while the other principal components correspond to the local spectral variability of the pixels in the homogeneous area.

The determination of can be achieved by comparing the size of the characteristic value. Set a threshold , if the cumulative eigenvalues corresponding to the first pivotal elements account for the percentage of the total eigenvalues to reach the threshold:

(9)

Therefore, the first principal components can be calculated by the first eigenvectors,

(10)

Then, the reconstructed spectral library by dominant characteristic components is termed as the endmember library,

(11)

Similarly, the next principal components are constructed as the spectral variability library as

(12)

Iii-C Spectral Variability Augmented Sparse Unmixing (SVASU)

Fig. 1 displays the flowchart of proposed SVASU where the well-known Jasper data set is taken as an example. The proposed SVASU adopts a two-order decomposition structure, in which the first order is to decompose the hyperspectral image data into the product of endmember library and abundance fractions roughly, the second order is to further represent the reconstruction error of pixels from first order as the influence of spectral variability library.

Let be the spectral variability library containing spectral variability signatures, and denotes the coefficients corresponding to library . Based on the objective function in Eq. (6), an extra spectral variability augmented data fitting term is introduced in the proposed SVASU model, along with the coefficient generalization regularized term, which is expressed as follows,

(13)

where balances the contribution of data fitting from both spectral spectra and spectral variability, and controls the sparsity of abundance and the generalization of spectral variability coefficient.

Following [55], we relax the term by . Thus the function in (13) can be written as

(14)

where is a diagonal matrix with the -th diagonal element formulated as follows, in which is a stabilization parameter,

(15)

In order to integrate non-negative constraint conditions into the objective function , we introduce Lagrangian multipliers to restrict .Therefore, the function (14) is equivalent to the following function:

(16)

When is fixed, the original problem can be turned into a general convex optimization problem. To update all matrixes of the objective function, we use an alternating projected gradient method. By setting the partial derivatives of with respect to and respectively, we obtain the following function:

(17)

We take advantage of KKT conditions in this mathematical optimization problem. Therefore, the following condition should be satisfied:

(18)

Then, we design the following update rules. The convergence of two update rules will be analyzed in the APPENDIX.

(19)

Therefore, an iterative procedure is adopted. In each iteration, is calculated with the current and , and then and are updated based on the current calculated . The iterative procedure is repeated until the algorithm converges. The pseudo code of SVASU is illustrated in Algorithm 1.

1:Hyperspectral data matrix ; Observed image spectral library ; Spectral variability library .
2:Abundance fraction and variability coefficient .
3:Initialize and randomly; ;
4:Repeat:
5:Update ;
6:Update ;
7:Update ;
8:;
9:Until Convergence;
10:Return and ;
Algorithm 1 SVASU.

Iv Experiments

In this section, a series of experiments on both the simulated data set and two real-world data sets (Jasper and Cuprite data sets) is designed to demonstrate the effectiveness of SVASU algorithm for HSIs. Several well-known sparse unmixing algorithms are adopted for comparison, including SUnSAL [9], CLSUnSAL [16], ADSpLRU [26], JSpBLRU [27], MUA [23]. For all these methods, the coefficient matrices corresponding to each spectral library are all randomly initialized and the values of parameters in each algorithm refer to the reported optimal values.

Iv-a Evaluation Metrics

To quantitatively evaluate the performance of our proposed SVASU algorithm, the two types of pixel reconstruction error are employed as in the experiments. Firstly, the root mean square error (RMSE) is used to measure the distance between the true signal and its estimated value, like that of image pixel and its reconstructed version as , and true abundance and its estimated version as , which are defined as,

(20)
(21)

Furthermore, the signal-to-reconstruction-error (SRE) is also adopted to measure the quality of the reconstruction of signals by different algorithms, which can demonstrate more information regarding the power of the error in relation to the power of the signal. Generally, the higher the SRE (dB) value, the better the performance of the algorithm. Similarly, we definite and as follows,

(22)
(23)
(a) Endmember Library
(b) Spectral Variability Library
Fig. 2: The Endmember Library and Spectral Variability Library obtained of the Synthetic data set through SVASU.
Fig. 3: The performances of SVASU with respect to parameters (a) , and (b) in terms of SRE and RMSE for both abundance and pixels on the synthetic data set.
SUnSAL CLSUnSAL JSpBLRU ADSpLRU MUA SVASU
SRE 8.0104 8.0026 7.1616 8.0026 18.9487 34.3370
RMSE 1.2232 1.2241 1.3436 0.0608 0.5680 0.0573
SRE 0.2409 0.2220 1.0789 0.1460 0.2334 1.7884
RMSE 0.1136 0.1133 0.0901 0.1129 0.1136 0.0732
TABLE I: Average SRE value and RMSE value obtained by comparison algorithms on the Synthetic data set. Note that the best results are in bolded.

Iv-B Experiments Over Synthetic Data set

In this section, the simulated data are generated in the way similar to [56] and [57] based on the LMM in (1). The synthetic data set has pixels. The endmember spectral signatures are using five spectra of minerals from the U.S. Geological Survey (USGS) containing as many as 420 bands covering from 400 to 2500 nm. Then, 30 dB Gaussian noise is added to each signature to simulate spectral variability. Considering that each pixel is unlikely to have a large number of endmembers in real scenarios (typically, less than 5), the maximum number of active endmembers per pixel is set to 4 to ensure abundance sparsity. Furthermore, the abundance vectors corresponding to active endmembers are generated using a mixture of uniform Dirichlet distributions [58]. Finally, 40 dB Gaussian noise is added to the whole image. In this experiment, the In-situ spectral library is simply composed of the selected five spectra from USGS and their corresponding spectra owning spectral variability. After the library segmentation by PCA, we acquire the (a) endmember library and (b) spectral variability library as shown in Fig. 2.

First of all, the impact of parameters, including , , and , are discussed through several sets of simulated experiments. The curves of these parameters of our method are shown in Fig. 3

(a) and (b). The evaluation metrics adopted here are the average SRE and RMSE calculated by (

20)-(23). On one hand, in Fig. 3(a), is preset to 10 and and vary in and exponentially. The proposed SVASU algorithm achieves a desired result when and , with lower RMSE value and higher SRE value both for abundance and pixels. On the other hand, Fig. 3(b) displays the performances of SRE and RMSE under different values of parameter tend to prominent when reaches 9. Thus, the parameter , , and are respectively set to be 9, 10, and in the experiments on synthetic data set.

Table I shows the average SRE and RMSE value between the ground-truth pixels or abundance and corresponding matrixes acquired by different algorithms over synthetic data set. The performance of the proposed SVASU algorithm outperforms other methods on the estimation of either fractional abundance or pixel reconstruction.

Iv-C Experiments Over Jasper Data set

The real-world Jasper data set is collected by an airborne visible/infrared imaging spectrometer (AVIRIS) sensor111Data available online at https://rslab.ut.ac.ir/data., which consists of 224 spectral bands ranging from 380 to 2500 nm and the spectral resolution is up to 9.46 nm. The size of this scene is 512614. In the experiments, a subimage of 100100 pixels is used. Due to dense water vapor and atmospheric effects, there exists some noisy bands, including the bands 13, 108112, 154166, and 220224. After removing these bands, a total of 198 reflectance bands are finally adopted. There are four endmembers in the selected region, including road, soil, water, and tree, and their corresponding ground-truth abundance are shown in Fig. 4.

Fig. 4: The reference four endmembers and their corresponding abundance of the Jasper data set.
(a) Endmember Library
(b) Spectral Variability Library
Fig. 5: The Endmember Library and Spectral Variability Library obtained of the Jasper data set through SVASU.
SUnSAL CLSUnSAL JSpBLRU ADSpLRU MUA SVASU
SRE 15.8183 16.9954 18.7103 17.2011 18.6439 19.5641
RMSE 0.0986 0.1083 0.1024 0.1055 0.0789 0.0753
SRE 3.9046 4.1053 2.9957 2.6846 6.7230 8.1455
RMSE 0.1150 0.0968 0.1161 0.1224 0.0796 0.0627
TABLE II: Average SRE value and RMSE value obtained by comparison algorithms on the Jasper data set. Note that the best results are in bolded.
Fig. 6: Visual analysis of the proposed SVASU model conducted on Jasper data set.

As shown in Fig. 1, the In-situ spectral library obtained by SPEE is shown in the first column, and then after the segmentation step described in Section III. B, we get the endmember library and spectral variability library shown in Fig. 5. To analyze the impact of three regularization parameters, including , , and , we first fix at 10 to discuss the performance of SVASU under different values of and . As shown in Fig. X(a), with the increase in value, the performances of abundance estimation and pixel reconstruction demonstrate different trends of development. Since the gap between the maximum and minimum values of evaluation matrixes for pixel reconstruction is small, the optimal parameter is set to and is set to corresponds to good capability of abundance estimation. After that, we further discuss the range of as shown in Fig. X(b). Through observation, it can be found that when the value of is lower than 3, the unmixing results of both abundance accuracy and data fidelity are superior to other cases. Thus, the optimal value of is set to 1.

Table II shows the average SRE and RMSE values between the ground-truth pixels or abundance and corresponding matrixes acquired by different algorithms over synthetic data set. The performance of the proposed SVASU algorithm outperforms other methods on the estimation of fractional abundance. However, due to the second level decomposition by our extracted spectral variability, the performance of pixel reconstruction is not as well as other sparse unmixing methods as expected.

Furthermore, to carefully analyze the results of two-order reconstruction, Fig. 6 depicts the visual results of the important items. Each visual result is averaged over each band, including the image data , the first order reconstruction item , the second order reconstruction item , the first order reconstruction error item , and the second order reconstruction error item . It can be observed that on the Jasper data set, the amount of information occupied by is probably twice as much information as that represents by . Moreover, through the second order reconstruction by spectral variability library, the image reconstruction error reduced to a tenth on Jasper data set, which completely proves the validity and rationality of the proposed SVASU method.

Iv-D Experiments Over Cuprite Data set

Fig. 7: Visual analysis of the proposed SVASU model conducted on Cuprite data set.

The hyperspectral data set named Cuprite data set, acquired by AVIRIS222http://aviris.jpl.nasa.gov/html/aviris.freedata.html., is also adopted in real experiments, which contains 224 bands ranging from 370 to 2510 nm with a ground instantaneous field of view of 20 m. The cropped image corresponds to a pixel subset of the sector labeled as in the online data. After removing noisy bands and water-absorption bands (including bands , , , and ), a total of 186 reflectance bands are finally adopted. The optimal parameters of proposed SVASU method are , , and .

Given that there is no available ground-truth in terms of abundances and endmembers of Cuprite data set, only visual results of two-order reconstruction in SVASU are shown as Fig. 7. Similarly, each visual result is averaged over each band, including the two-order reconstruction item , , and the two-order reconstruction error item , . It can be observed that the amount of information occupied by is more than twice as much information as that by . This also confirms that in the processing of , the main components are constructed by endmember library while the other components are constructed as the spectral variability library. Moreover, through the second order reconstruction by spectral variability library, the image reconstruction error reduced to less than a tenth on Cuprite data set. Therefore, the effectiveness of the proposed SVASU method are confirmed on the real-world data set.

V Conclusion

In this paper, a spectral variability augmented sparse unmixing model (SVASU) is proposed, which provides a new way to construct spectral variability explicitly from the spectral library for sparse unmixing. It is noted that a in-situ spectral library acquired under spatial purity is separated into the endmember library and the variability library by PCA. Afterwards, the proposed SVASU adopts a two-order decomposition structure to perform SU, in which the first order is to decompose the hyperspectral image data into the product of endmember library and abundance fractions roughly, the second order is to further represent the reconstruction error of pixels from first order as the influence of spectral variability library. Experimental results over both synthetic and real-world datasets demonstrate that the proposed SVASU model can certainly improve the performance of spectral linearly unmixing and through the introducing of second order reconstructed loss, the ability of the proposed framework to model the spectral variability is improved. Meanwhile, the use of those smoothness terms guarantees a moderate variation of the spectral variability.

Appendix A Proof of Convergence of SVASU

The problem in (14) contains two unknown matrices, and we design different rules to alternately update one by fixing the other. Following [59, 60], auxiliary function approach is used to prove the convergence of the proposed update rules. Function is an auxiliary function for if the conditions

(24)

are satisfied for any . Then, if

(25)

it has been proven that the following inequalities held:

(26)

Thus, the function is monotonically decreasing.

A-1 Fixing , minimizing

First, when is fixed, the minimization problem in (14) can be written as

(27)

in which . For the first two terms in (27), the following inequality holds [61].

(28)
(29)

where , and . It is obvious that . Taking as an example, can be proved in detail as follows,

(30)

Thus, and can be selected as the auxiliary functions of and correspondingly.

Then, for the last term , the following auxiliary function is constructed as,

(31)

It is straightforward to verify that . To show that , we have the following calculation,

(32)

Thus, can be an auxiliary function of .

Finally, is an auxiliary function of in (14). Currently, according to Eq. (25), we find the minimum of by fixing , which is given by

(33)

Then, we can get

(34)

A-2 Fixing , minimizing

When is fixed, the minimization problem in (14) can be written as

(35)

in which .

Similarly, the following two inequalities hold:

(36)
(37)

in which , and .

Thus, is an auxiliary function of in Eq. (35). Currently, according to Eq. (25), we find the minimum of by fixing , which is given by

(38)

Solving for , the minimum is

(39)

Thus, updating and alternatively by using the rules in (34) and (39) can guarantee that the objective function in (14) is non-increasing.

References

  • [1] P. Ghamisi, N. Yokoya, J. Li, W. Liao, S. Liu, J. Plaza, B. Rasti, and A. Plaza, “Advances in hyperspectral image and signal processing: A comprehensive overview of the state of the art,” IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 4, pp. 37–78, 2017.
  • [2] J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, and J. Chanussot, “Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 2, 2012.
  • [3] D. Heinz and Chein-I-Chang, “Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 3, pp. 529–545, 2001.
  • [4] R. Borsoi, T. Imbiriba, J. C. Bermudez, C. Richard, J. Chanussot, L. Drumetz, J.-Y. Tourneret, A. Zare, and C. Jutten, “Spectral variability in hyperspectral data unmixing: A comprehensive review,” IEEE Geoscience and Remote Sensing Magazine, pp. 2–49, 2021.
  • [5] M. Winter, “N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data,” in Proceedings of SPIE, Image Spectrometry V, vol. 3753, 1999, pp. 266–277.
  • [6] J. Boardman, F. Kruse, and R. Green, “Mapping target signatures via partial unmixing of aviris data,” in Summaries of JPL Airborne Earth Science Workshop, 1995, pp. 23–26.
  • [7] J. Nascimento and J. Dias, “Vertex component analysis: a fast algorithm to unmix hyperspectral data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 4, pp. 898–910, Apr. 2005.
  • [8] J. Li, A. Agathos, D. Zaharie, J. M. Bioucas-Dias, and X. Li, “Minimum volume simplex analysis: A fast algorithm for linear hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 5067–5082, 2015.
  • [9] M. D. Iordache, J. M. Bioucas-Dias, and A. Plaza, “Sparse unmixing of hyperspectral data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 6, pp. 2014–2039, 2011.
  • [10]

    S. Mei, K. Yan, M. Ma, X. Chen, S. Zhang, and Q. Du, “Remote sensing scene classification using sparse representation-based framework with deep feature fusion,”

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 5867–5878, 2021.
  • [11] X. Xu, B. Pan, Z. Chen, Z. Shi, and T. Li, “Simultaneously multiobjective sparse unmixing and library pruning for hyperspectral imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 4, pp. 3383–3395, 2021.
  • [12] M. Ma, S. Mei, S. Wan, Z. Wang, Z. Ge, V. Lam, and D. Feng, “Keyframe extraction from laparoscopic videos via diverse and weighted dictionary selection,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, pp. 1686–1698, 2021.
  • [13] S. Mei, M. Ma, S. Wan, J. Hou, Z. Wang, and D. D. Feng, “Patch based video summarization with block sparse representation,” IEEE Transactions on Multimedia, vol. 23, pp. 732–747, 2021.
  • [14] L. Qi, J. Li, Y. Wang, Y. Huang, and X. Gao, “Spectral cspatial-weighted multiview collaborative sparse unmixing for hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 12, pp. 8766–8779, 2020.
  • [15] Z. Zhang, S. Liao, H. Fang, H. Zhang, and S. Wang, “Hyperspectral unmixing using spectral library sparse scaling and guided filter,” IEEE Geoscience and Remote Sensing Letters, pp. 1–5, 2020.
  • [16] M.-D. Iordache, J. M. Bioucas-Dias, and A. Plaza, “Collaborative sparse regression for hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 1, pp. 341–354, 2014.
  • [17] J. M. Bioucas-Dias and M. A. T. Figueiredo, “Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing,” in 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, 2010, pp. 1–4.
  • [18] T. Ince, “Double spatial graph laplacian regularization for sparse unmixing,” IEEE Geoscience and Remote Sensing Letters, pp. 1–5, 2021.
  • [19] M.-D. Iordache, J. M. Bioucas-Dias, and A. Plaza, “Total variation spatial regularization for sparse hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 11, pp. 4484–4502, 2012.
  • [20] J. Qin, H. Lee, J. T. Chi, L. Drumetz, J. Chanussot, Y. Lou, and A. L. Bertozzi, “Blind hyperspectral unmixing based on graph total variation regularization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 4, pp. 3338–3351, 2021.
  • [21]

    X.-R. Feng, H.-C. Li, S. Liu, and H. Zhang, “Correntropy-based autoencoder-like nmf with total variation for hyperspectral unmixing,”

    IEEE Geoscience and Remote Sensing Letters, pp. 1–5, 2020.
  • [22] S. Zhang, J. Li, H.-C. Li, C. Deng, and A. Plaza, “Spectral cspatial weighted sparse regression for hyperspectral image unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 6, pp. 3265–3276, 2018.
  • [23] R. A. Borsoi, T. Imbiriba, J. C. M. Bermudez, and C. Richard, “A fast multiscale spatial regularization for sparse hyperspectral unmixing,” IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 4, pp. 598–602, 2019.
  • [24] J. Li, Y. Li, R. Song, S. Mei, and Q. Du, “Local spectral similarity preserving regularized robust sparse hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 10, pp. 7756–7769, 2019.
  • [25] L. Sun, F. Wu, C. He, T. Zhan, W. Liu, and D. Zhang, “Weighted collaborative sparse and l1/2 low-rank regularizations with superpixel segmentation for hyperspectral unmixing,” IEEE Geoscience and Remote Sensing Letters, pp. 1–5, 2020.
  • [26] P. V. Giampouras, K. E. Themelis, A. A. Rontogiannis, and K. D. Koutroumbas, “Simultaneously sparse and low-rank abundance matrix estimation for hyperspectral image unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 8, pp. 4775–4789, 2016.
  • [27] J. Huang, T.-Z. Huang, L.-J. Deng, and X.-L. Zhao, “Joint-sparse-blocks and low-rank representation for hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 4, pp. 2419–2438, 2019.
  • [28] H. Li, R. Feng, L. Wang, Y. Zhong, and L. Zhang, “Superpixel-based reweighted low-rank and total variation sparse unmixing for hyperspectral remote sensing imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, pp. 629–647, 2021.
  • [29] L. Drumetz, J. Chanussot, and C. Jutten, “Spectral unmixing: A derivation of the extended linear mixing model from the hapke model,” IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 11, pp. 1866–1870, 2020.
  • [30] L. Drumetz, J. Chanussot, C. Jutten, W.-K. Ma, and A. Iwasaki, “Spectral variability aware blind hyperspectral image unmixing based on convex geometry,” IEEE Transactions on Image Processing, vol. 29, pp. 4568–4582, 2020.
  • [31] J. Theiler, A. Ziemann, S. Matteoli, and M. Diani, “Spectral variability of remotely sensed target materials: Causes, models, and strategies for mitigation and robust exploitation,” IEEE Geoscience and Remote Sensing Magazine, vol. 7, no. 2, pp. 8–30, 2019.
  • [32] R. A. Borsoi, T. Imbiriba, and J. C. M. Bermudez, “A data dependent multiscale model for hyperspectral unmixing with spectral variability,” IEEE Transactions on Image Processing, vol. 29, pp. 3638–3651, 2020.
  • [33] T. Uezato, M. Fauvel, and N. Dobigeon, “Hyperspectral unmixing with spectral variability using adaptive bundles and double sparsity,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 6, pp. 3980–3992, 2019.
  • [34] L. Drumetz, T. R. Meyer, J. Chanussot, A. L. Bertozzi, and C. Jutten, “Hyperspectral image unmixing with endmember bundles and group sparsity inducing mixed norms,” IEEE Transactions on Image Processing, vol. 28, no. 7, pp. 3435–3450, 2019.
  • [35] A. Halimi, N. Dobigeon, and J.-Y. Tourneret, “Unsupervised unmixing of hyperspectral images accounting for endmember variability,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 4904–4917, 2015.
  • [36] H. Liu, Y. Lu, Z. Wu, Q. Du, J. Chanussot, and Z. Wei, “Bayesian unmixing of hyperspectral image sequence with composite priors for abundance and endmember variability,” IEEE Transactions on Geoscience and Remote Sensing, pp. 1–15, 2021.
  • [37]

    Y. Zhou, E. B. Wetherley, and P. D. Gader, “Unmixing urban hyperspectral imagery using probability distributions to represent endmember variability,”

    Remote Sensing of Environment, vol. 246, p. 111857, 2020.
  • [38] P.-A. Thouvenin, N. Dobigeon, and J.-Y. Tourneret, “Hyperspectral unmixing with spectral variability using a perturbed linear mixing model,” IEEE Transactions on Signal Processing, vol. 64, no. 2, pp. 525–538, 2015.
  • [39] L. Drumetz, M.-A. Veganzones, S. Henrot, R. Phlypo, J. Chanussot, and C. Jutten, “Blind hyperspectral unmixing using an extended linear mixing model to address spectral variability,” IEEE Transactions on Image Processing, vol. 25, no. 8, pp. 3890–3905, 2016.
  • [40] D. Hong, N. Yokoya, J. Chanussot, and X. X. Zhu, “An augmented linear mixing model to address spectral variability for hyperspectral unmixing,” IEEE Transactions on Image Processing, vol. 28, no. 4, pp. 1923–1938, 2019.
  • [41] C. Zhang, L. Ma, J. Chen, Y. Rao, Y. Zhou, and X. Chen, “Assessing the impact of endmember variability on linear spectral mixture analysis (lsma): A theoretical and simulation analysis,” Remote Sensing of Environment, vol. 235, pp. 1–21, 11 2019.
  • [42] L. Drumetz, J. Chanussot, and C. Jutten, “Variability of the endmembers in spectral unmixing: Recent advances,” in 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2016, pp. 1–5.
  • [43] D. A. Roberts, M. Gardner, R. Church, S. Ustin, G. Scheer, and R. Green, “Mapping chaparral in the santa monica mountains using multiple endmember spectral mixture models,” Remote sensing of environment, vol. 65, no. 3, pp. 267–279, 1998.
  • [44] M. Xu, L. Zhang, B. Du, and L. Zhang, “An image-based endmember bundle extraction algorithm using reconstruction error for hyperspectral imagery,” Neurocomputing, vol. 173, pp. 397–405, 2016.
  • [45] F. Chen, K. Wang, and T. F. Tang, “Spectral unmixing using a sparse multiple-endmember spectral mixture model,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 10, pp. 5846–5861, 2016.
  • [46] C. Zhang, “Multiscale quantification of urban composition from eo-1/hyperion data using object-based spectral unmixing,” International journal of applied earth observation and geoinformation, vol. 47, pp. 153–162, 2016.
  • [47] R. A. Borsoi, T. Imbiriba, J. C. M. Bermudez, and C. Richard, “Deep generative models for library augmentation in multiple endmember spectral mixture analysis,” IEEE Geoscience and Remote Sensing Letters, pp. 1–5, 2020.
  • [48] M. Berman, L. Bischof, R. Lagerstrom, Y. Guo, J. Huntington, P. Mason, and A. A. Green, “A comparison between three sparse unmixing algorithms using a large library of shortwave infrared mineral spectra,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 6, pp. 3588–3610, 2017.
  • [49] K. D. Singh, D. Ramakrishnan, and L. Mansinha, “Relevance of transformation techniques in rapid endmember identification and spectral unmixing: A hypespectral remote sensing perspective,” in 2012 IEEE International Geoscience and Remote Sensing Symposium, 2012, pp. 4066–4069.
  • [50] Y. Shao and J. Lan, “A spectral unmixing method by maximum margin criterion and derivative weights to address spectral variability in hyperspectral imagery,” Remote Sensing, vol. 11, no. 9, p. 1045, 2019.
  • [51] F. A. Mianji and Y. Zhang, “Svm-based unmixing-to-classification conversion for hyperspectral abundance quantification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 11, pp. 4318–4327, 2011.
  • [52] T. Uezato, R. J. Murphy, A. Melkumyan, and A. Chlingaryan, “A novel spectral unmixing method incorporating spectral variability within endmember classes,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 5, pp. 2812–2831, 2016.
  • [53]

    S. Mei, M. He, Z. Wang, and D. Feng, “Mixture Analysis by Multichanel Hopfield Neural Network,”

    IEEE Geoscience and Remote Sensing Letters, vol. 7, pp. 455–459, 2010.
  • [54] S. Mei, M. He, Z. Wang, and D. Feng, “Spatial purity based endmember extraction for spectral mixture analysis,” IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 9, pp. 3434–3445, 2010.
  • [55]

    F. Nie, H. Huang, X. Cai, and C. Ding, “Efficient and robust feature selection via joint

    -norms minimization,” Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems (NIPS), vol. 23, pp. 1813–1821, 2010.
  • [56] L. Zhuang, C.-H. Lin, M. A. T. Figueiredo, and J. M. Bioucas-Dias, “Regularization parameter selection in minimum volume hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 12, pp. 9858–9877, 2019.
  • [57] J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, “Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 2, pp. 354–379, 2012.
  • [58] J. M. P. Nascimento and J. M. Bioucas-Dias, “Hyperspectral unmixing based on mixtures of dirichlet components,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 3, pp. 863–878, 2012.
  • [59] D. Lee and H. Seung, “Algorithms for non-negative matrix factorization,” International Conference on Neural Information Processing Systems, pp. 535–541, 2001.
  • [60] R. Huang, X. Li, and L. Zhao, “Spectral cspatial robust nonnegative matrix factorization for hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 10, pp. 8235–8254, 2019.
  • [61] F. Wang, T. Li, W. Xin, S. Zhu, and C. Ding, “Community discovery using nonnegative matrix factorization,” Data Mining and Knowledge Discovery, vol. 22, no. 3, pp. 493–521, 2011.