I Introduction
Hyperspectral imaging is a continuously growing field of study and has received considerable attention in the last decade. Hyperspectral data provide a wide spectral range, coupled with a high spectral resolution. These characteristics are suitable for the detection and classification of surfaces and chemical elements in the observed images. Rich information in the spectral dimension provides solutions to many problems that cannot be solved by traditional RGB imaging or multispectral imaging. However, because of factors such as the low spatial resolution of the spectral imaging devices, and the diversity of materials in a scene, an observed pixel may contain several materials. Spectral unmixing plays an important role in hyperspectral data processing, aimed at separating a pixel spectrum into a set of spectral signatures, termed endmembers, and a set of fractional abundances associated with each endmember [2].
A significant amount of effort has been made in the past decade to solve the spectral unmixing problem. Endmember identification and abundance estimation can be conducted either in a sequential or a simultaneous manner. Sequential techniques first determine the endmembers, endmember extraction algorithms such as purity index algorithm (PPI)
[3], vertex component analysis (VCA) [4] and NFINDR algorithm [5]. In the sequel, the problem boils down to determination of the fractional abundances [6]. Simultaneous techniques determine the endmembers and the estimated abundance fractions at the same time, mainly by using matrix decomposition methods [7], sparse learning methods [8], or optimization methods [9] to this purpose. Moreover, several other facts have been further taken into account to enhance the modeling capacity and improve the performance of the unmixing algorithms. As hyperspectral images inherently contain twodimensional information, it is natural to impose the spatial continuity on the estimated abundances [10, 11]. Considering the limitation of the linear unmixing due to the multilayer reflectance and the intimate interaction of materials, several works have investigated the nonlinear models and nonlinear unmixing algorithms [12, 13, 14, 15]. Taking into account the endmember variability in the observed scenario, researchers have considered a bundle of spectra to be signatures of a material during the unmixing process [16, 17, 18].In contrast to the increasing development of various types of unmixing algorithms, the way to perform an effective and objective evaluation of these algorithms remains restrictive and unconvincing, although both synthetic data and real data are used in the literature. Compared with the target detection and classification tasks where several labeled datasets are available, the lack of groundtruth information for the unmixing makes it difficult to evaluate and compare the algorithms in an objective manner. On one hand, synthetic data are generated by the assumed mathematical models. Validating an algorithm with the data generated by the same model restricts the evaluation of the general performance of the algorithm. On the other hand, as there is no groundtruth dataset for the spectral unmixing task, it is difficult to interpret the unmixing results of the algorithms even though they have been applied to real datasets. This restricts a comparison of the results on the basis of the observation and the intuitive interpretation of the estimated abundance maps. See Sec. IIB for a more detailed review of and discussion on this issue.
Many hyperspectral datasets are captured by airborne devices and satellites. Labeling these data with material distribution fractions is extremely difficult. In this work, we design several experimental scenes in our laboratory with controlled settings, where the pure material spectra and the material composition methods are known. We then capture these scenes with a hyperspectral device and perform the necessary preprocessing to generate the dataset. To the best of our knowledge, this dataset is the first publicly available dataset with groundtruth for spectral unmixing created in a systematic manner. A summary of our contributions and the main properties of the data as follows:

We build a laboratorycreated dataset with groundtruth for evaluating hyperspectral unmixing algorithms. This dataset is available on the corresponding author’s website http://www.jiechen.com/.

We design several scenes to model different mixture mechanisms, including printed checkerboards, mixed quartz sands, and reflection with a vertical board. Both the images of pure materials and the composition of each material are provided so that researchers have access to the endmembers and the corresponding abundances. The endmember variability and nonlinear mixture effects can also be observed with the created data. See Sec. III for further details on the settings.

We apply several typical unmixing techniques to these data. We find that linear unmixing algorithms result in abundance RMSEs of the order of for checkerboardtype data, and the nonlinear unmixing algorithms such as KHype and Hapke are beneficial for intimately mixed materials.
Note that this laboratorycreated dataset does not have the exact same properties as real airborne and satellite data. However, it is expected to help in reducing the gap between the theoretical research and practice, and to help researchers to evaluate their unmixing algorithms in a quantifiable and comparable manner.
The rest of this paper is organized as follows. Section II reviews several popular spectral mixing models, and presents the status and limitations of the current methods of evaluating the unmixing performance. Section III presents the properties, capturing settings, and designs scenes of our created dataset in detail. In order to validate our dataset, we apply several typical endmember extraction algorithms and unmixing techniques to the created scenes; this is described in Section IV. Section V concludes the paper and presents the discussion.
Notation. Normal font and denote scalars. Boldface small letters
denote vectors. All vectors are column vectors. Boldface capital letters
denote matrices. Specifically, let be an observed column pixel, supposed to be a mixture of endmember spectra, with the number of spectral bands. Assume that is the target endmember matrix, where each column is an endmember spectral signature. For the sake of convenience, we shall denote by the th row of , that is, the vector of the endmember signatures at the th wavelength band. Let be the abundance column vector associated with the pixel .Ii Existing mixing models and unmixing performance evaluation
In this section, we will briefly review several widely used mixing models and summarize the usual methods of evaluating the unmixing performance in the literature. We shall see that these models and restrictions in the existing evaluation methodology considerably motivate the experimental settings in this work.
Iia Widely used mixture models
IiA1 Linear mixture model
The linear mixture model (LMM) is widely used to identify and quantify pure components in remotely sensed images due to its simple physical interpretation and trackable estimation process. If the multiple scattering among distinct endmembers is negligible and distinct materials are considered to be partitioned in a checkerboard manner, the spectrum of a low resolution pixel is approximated by a linear mixture of endmember spectra weighted by the corresponding fractional abundances[2][19][20], namely, the observed , is given by
(1) 
with being a modeling noise. Specifically, since components of represent fractions of each material, it is often considered that they satisfy the nonnegativity (ANC) and sumtoone constraints (ASC):
(2)  
(3) 
Although the linear mixture model has obvious practical advantages, there are many situations in which it may not be appropriate and can be advantageously replaced by a nonlinear one. Some typical ones are presented herebelow.
IiA2 Bilinear mixing model
Considering that multiple scattering effects can be observed on complex vegetated surfaces where it is assumed that incident solar radiation is scattered through multiple bounces involving several endmembers. Bilinear model accounts for presence of secondorder photon interactions by introducing additional interaction terms in the linear model. The model is given by:
(4) 
where denotes the elementwise product of two vectors. Relating and with different constraints leads a variety of bilinear models [12].
IiA3 Linear mixing/nonlinear fluctuation model
In our previous work[13], we do not restrict to use the extra polynomial terms to describe the nonlinearity as in the bilinear model. We assume that the mixing mechanism can be described by a linear mixture of endmember spectra, with additive nonlinear fluctuations that allows to take complex interactions of endmembers into account.
(5) 
where can be any realvalued functions on a compact , of a reproducing kernel Hilbert space with being its reproducing kernel. Selecting kernels allows to capture the different types of nonlinearity of the mixture model. Performing the unmixing with (6) suggests to simultaneously estimate the abundance and the nonlinear fluctuation with the socalled kernel trick. Introducing the abundance in the nonlinear term with a postnonlinear form, we further have
(6) 
This model was proposed in[21]. In[22][23], the term is also called the residual term and the associated algorithm is called the residual component analysis.
IiA4 Intimate model
Intimate mixtures are mixtures of grains or particles that are in close contact with each other. Light typically interacts multiple times with the particles making up the mixture before reaching the observer. The optical characteristics of such mixtures depend on many parameters, such as the total number and the fractions of each component, the grain size distribution of each component, the shape and orientation of the grains, the average optical distance between reflections, the absorption and scattering characteristics, etc. Model proposed by Hapke [24] is a popular model that describes the optical characteristics of intimately mixed materials. It can be viewed as a postnonlinear model applied to a linear mixture in the albedo domain. See [24] for more details. Though the intimate model has a clear physical interpretation, few works perform unmixing with this model due to its complexity and requirement of several parameters of the scene.
IiB Evaluation of the unmixing performance
In contrast to the increasing development of various types of unmixing algorithms, the way to perform an effective and objective evaluation of algorithms remains restrictive and unconvincing. Most works in the literature evaluate the unmixing performance with both synthetic data and real data, however, with significant limitations.
IiB1 Evaluation with synthetic data
Synthetic data are generated using an assumed mathematical model. The use of synthetic data is considered as a baseline to evaluate and compare the unmixing algorithms, as all information, including endmembers, abundances, and the mixing model, are known. Therefore we can evaluate algorithms with objective measures. However, it is clear that the use of synthetic data is very restrictive because of the following reasons:

Each model has its own limitation in modeling a real mixture in the observed scene. It is not trivial to predict to the performance of an algorithm (particularly nonlinear ones) in real applications, even though it works well with the synthetic data generated using a mathematical model.

It is common in the exiting literature to validate an algorithm with data generated by the same model. It is not convincing to evaluate the performance of an algorithm in such a manner. For instance, an algorithm carefully devised from the bilinear model may outperform other nonlinear unmixing algorithms when we use data generated by the same model, however, it does not make much sense if the performance of this algorithm is severely biased when the practical mixture is not ideally bilinear.
IiB2 Evaluation with real data
Besides using synthetic data, most of the previous studies also use real images such as the Cuprite dataset [13, 20, 14, 25, 26] to validate the proposed algorithm. However, people are currently faced with a dilemma of using these real datasets captured by airborne methods. On one hand, while considering the limitation of synthetic data, evaluating algorithms with real data is indeed important. On the other hand, it is very difficult to compare the obtained results because the lack of the associated groundtruth information for real data precludes a quantitative and objective evaluation of algorithms. Researchers make the following efforts to evaluate the unmixing performance while using real data:

Interpreting results with intuitive observation: When testing unmixing algorithms with real data, most of the existing works [21, 20, 12, 27, 26, 25, 28, 29] illustrate the results in a straightforward manner by showing the estimated abundance maps and making visually intuitive comparisons. Clearly, this qualitative method is not very helpful to understand the performance of these algorithms.

Comparing reconstruction errors: Besides an intuitive examination of abundance maps, the spectral reconstruction error (RE) and the spectral angle (SA) between the observed pixel and the reconstructed one are frequently used as quantitative measures in the literature [12, 23, 11, 28]. However, it is widely accepted that the quality of reconstruction is not necessarily in proportion to the quality of unmixing, particularly for real images where a complex nonlinear mixture is involved [13, 22, 14].

Comparing results with classification results: In contrast to the unmixing task, there exist several datasets with groundtruth labels for the hyperspectral classification. Some works benefit from this information to evaluate the unmixing result in an indirect manner. In [30, 11]
, the authors consider the abundance obtained from unmixing algorithms as the feature of a classifier, and then compare the classification results with the groundtruth. In
[10, 31, 32, 33, 29], the authors compare the abundance maps with the classification results obtained using the Tricorder software. 
Comparing results with those obtained from existing algorithms: A handful of works consider the results from a given unmixing algorithm as the standard reference, and then compare the performance of other algorithms with it. For instance, in [34], the authors use this methodology to generate “labeled” unmixing data. In [35, 36], the authors evaluate the results of NMF by comparing them with those of FCLS.
It is clear that none of the above methods provide a convincing quantitative performance evaluation of unmixing algorithms. The result illustration with real data in the existing works is therefore not highly informative and persuasive. This motivates us to create a proper dataset with groundtruth in our laboratory for a fair evaluation of unmixing algorithms.
Iii Description of the dataset
Iiia General description and properties of the dataset
Our dataset possesses the following properties:

A variety of mixing scenes: We design several different scenes to mimic various mixture models with experiment parameters being strictly controlled, including checkerboardtype data, mixed quartz sands, and reflection with a vertical board. A single scene consists of up to 4 materials and 14 different mixtures.

Available groundtruth: Both material endmembers and their associated abundances are known. We image pure materials for collecting the endmembers, and use the prior known compositions or highresolution images to provide the abundance fractions. With this information, both of the endmember extraction and the abundance estimation algorithms can be evaluated.

Spatial property: A raw image has a spatial resolution of pixels. In order to provide data with uniform illumination and remove unrelated objects (e.g., conveyer background, calibration board, the container), we keep the region of pixels from the center of each scene. Both raw images with considerably more pixels and the clipped images are provided in our dataset, where the clipped images already provide at least 3600 pixels with groundtruth.

Spectral property: The data have 256 channels in the spectral dimension, ranging from to , with a spectral resolution of up to . The noise of each band is not uniform. First and last several bands are much noisy due to the low sensitivity of the spectrometer at these two ends. These bands can simply be discarded, or kept for studying the band selection, robust unmixing, etc.
IiiB Methods
IiiB1 Hyperspectral device description and settings
Our data were collected by the GaiaField and GaiaSorter systems in our laboratory. Our GaiaField (Sichuan Dualix Spectral ImageTechnology Co. Ltd., GaiaFieldV10) is a pushboom imaging spectrometer with a HSIAOL50 lens, covering the visible and NIR wavelengths ranging from to , with a spectral resolution of up to . This camera adopts CCD instead of CMOS detector, leading to a lower noise level. Provided that the camera is not running in an environment with a very high temperature, the heating effect, readout noise and dark current noise will be stable. The dark current has a negligible effect for the visible and NIR range. GaiaSorter sets an environment that isolates external lights, and is endowed with a conveyer to move samples for the pushboom imaging. Conveyor belt is made of aluminum with the oxidizing blackened surface. It can be considered as a blackbody in the most range of . Four tungstenbromine lamps were used to form a hemisphericaldirectional illumination that simulated uniform realworld solar illumination. A halogen lamp is characterized by its stability in illumination. After a warmup time of , the spatial illumination uniformity of our lamps is greater than and the light sources are proved to be stable over time. The distance between the camera and the samples was adjustable. The exposure time, moving speed of the conveyer, and focusing were automatically computed and controlled by the associated software that was run on a computer.
In our experiments, the camera was right above the target. The distance between the lens and the samples was adjusted to , and the field angle of the camera was . Therefore, the width of view was . The vertical distance between the lamps and the samples was approximately . At these settings, a collected raw hyperspectral cube consisted of 348 and 455 pixels in width and height, and 256 spectral bands. We then found that the resolution of a collected image is /pixel. In our experiments, we controlled the crosssectional areas of each pure material to be less than , to ensure that each pixel was a mixture of multiple materials.
IiiB2 Data normalization
In order to convert the collected light intensity levels to reflectance values, we applied the blackwhite normalization to the raw data. This normalization also aimed to remove the effect of the dark current of the camera sensor and avoided the uneven light intensity of each band. In the offline phase, the black image is acquired by turning off the light source and covering the camera lens with its cap. The white image was acquired by imaging a standard white board (foamed PEFE resin, certified by China Metrology Institute) under the same condition as that of the raw image for calibration. Then, a normalized pixel was obtained by performing the following step on the raw observation [37]:
(7) 
where and denote the spectra of the black and white frames respectively, and the difference and the division operations are applied in an elementwise manner.
IiiC Experimental scenes and data description
IiiC1 Scene I
In the first scene, we aimed to design a scene that matched the linear mixing model. As in the linear model, the abundance can be seen as the area percentage of each material in a pixel; we used a checkerboard image to set this scene. Three typical colors of color printing, namely, magenta, yellow, and cyan, were used as the endmembers, as shown in Fig.
2(a). We printed these colors in a checkerboard manner consisting of a large number of or squares. The reference materials and the mixed scenes are printed on the same type of paper. This setting mimicked the low resolution data capture process and was consistent with the assumption of the linear mixture model. We prepared ten mixture settings in this way with the fraction abundances reported in Tab. I, and the mixture patterns depicted pictorially in Figs. 2(bk). The spectral curves of the endmembers and the mixtures are illustrated in Fig. 3.Magenta  Yellow  Cyan  

Mixture 1  0  
Mixture 2  0  
Mixture 3  0  
Mixture 4  0  
Mixture 5  0  
Mixture 6  0  
Mixture 7  
Mixture 8  
Mixture 9  
Mixture 10 
IiiC2 Scene II
In the second scene, we mimicked the intimate mixture case by using four different colors of quartz sands. The diameter of the granules was approximately
, which was controlled by filtering through 55mesh and 60mesh sieves. Using sands with a uniform size allowed us to focus on the volume fractions by ruling out the factors such as the crosssectional areas of the granules. We first prepared fourteen mixtures where the sands were uniformly distributed as shown in Fig.
4. The percentage of each color of sands was considered as the fractional abundances with the value reported in Tab. II. The spectral curves of the endmembers and the mixtures are illustrated in Fig. 5. Thereafter, we prepared four mixtures with spatial patterns, as shown in Fig. 6. Unlike in the other experiments where abundances were given on the basis of the material composition, in this experiment, the material composition of each pixel couldnot be known immediately at this setting. Therefore, we captured highresolution RGB images, and then analyzed the percentage of each colored sand in a lowresolution hyperspectral image with the help of the associated resolution RGB image^{1}^{1}1We captured hyperspectral images along with their associated highresolution RGB images, and aligned them by the preset marks. Consequently, a high resolution RGB region could be linked to a hyperspectral pixel using the spatial resolution ratio. Then the proportion of the materials in a hyperspectral pixel can be evaluated by analyzing the RGB region.. This setting allowed to test algorithms that took into account the spatial correlation.Red  Green  Blue  White  

Mixture 1  
Mixture 2  
Mixture 3  
Mixture 4  
Mixture 5  
Mixture 6  
Mixture 7  
Mixture 8  
Mixture 9  
Mixture 10  
Mixture 11  
Mixture 12  
Mixture 13  
Mixture 14 
IiiC3 Scene III
In this scene, we aimed to create an environment in which the secondorder reflection exists. The color squares prepared in Scene I were used. Further, a plastic board was set beside the printed squares to reflect the light. A diagram of this scene is shown in Fig. 7. Eight mixtures were captured in this scene, including five mixtures with reflection, and three pure materials with reflection. The mixture fractions are reported in Tab. III, and the spectral curves of the endmembers and the mixtures are illustrated in Fig. 8.
Magenta  Yellow  Cyan  

Mixture 1  
Mixture 2  
Mixture 3  
Mixture 4  0  
Mixture 5  
Mixture 6  0  
Mixture 7  1  
Mixture 8  0 
Iv Algorithm tests with our dataset
In this section, we apply several typical unmixing algorithms to our dataset and examine the unmixing results. Note that the purpose of this application is to check how some existing algorithms perform with these data, rather than to provide an optimal algorithm. As it is straightforward to extract the endmember from the pure materials, we particularly focus on the unmixing with known endmembers and calculate the root mean square error (RMSE), namely,
(8) 
to evaluate the abundance estimation performance of the above unmixing algorithms, where is the total number of pixels, and denote the true and estimated abundance vectors of the th pixel. Endmember extraction is also tested in the linearly mixed Scene I, as to be seen later.
Iva Results of Scene I
IvA1 Abundance estimation results of Scene I
We extracted the endmembers from the pure color blocks in Fig. 2, and then applied the fully constrained least square method (FCLS), and the nonnegative constrained least square method (NCLS) to the mixtures. The FCLS method relies on the linear mixture model and minimizes the leastsquare error, subject to the nonnegativity and the sum to one constraints. By relaxing the sumtoone constraint, the solution leads to the NCLS method. The mean estimates and RMSE results of FCLS and NCLS are showed in Tab. IV. We observe that both the algorithms resulted in RMSE values of the order of and the two algorithms exhibited a comparable performance. The histograms of the estimates of Mixture 2 are shown in Fig. 9.
FCLS  NCLS  

Magenta  Yellow  Cyan  RMSE  Magenta  Yellow  Cyan  RMSE  
Mixture 1  0.16%  0.34%  0.0749  
Mixture 2  0.03%  0.0155  0  
Mixture 3  0.0493  
Mixture 4  0.0237  
Mixture 5  0.0198  
Mixture 6  %  0.0239  %  
Mixture 7  0.0263  
Mixture 8  0.0275  
Mixture 9  0.0221  
Mixture 10  0.0238 
IvA2 Endmember extraction results of Scene I
We also applied several typical endmember extraction algorithms to the mixture shown in Figs. 2(bk) and chose two widely used metrics, namely spectral angle distance (SAD) and spectral information divergence (SID), to evaluate the similarity between a real endmember and its estimate. The following methods we tested: VCA [4], NFINDR [5], MVCNMF [7], MVSA [9], and SPICE [38]. Among these algorithms, the VCA and NFINDR required the existence of pixels of pure materials, and the rest of them did not impose this requirement. Note that the scenes shown in Figs. 2(bk) did not in fact contain pure color blocks.
The SAD and SID between the real endmember spectral signature and its estimate are respectively given by: and
with the probability distribution vector with each endmember signature
, and its estimate . The SAD and SID results of different endmember extraction methods are shown in Tab. V.SAD  SID  

VCA  NFINDR  MVCNMF  MVSA  SPICE  VCA  NFINDR  MVCNMF  MVSA  SPICE  
Mixture 1  10.1101  11.2335  10.1722  10.1101  9.9481  0.0527  0.0676  0.0529  0.0527  0.0517 
Mixture 2  15.0557  16.0560  15.1569  15.0557  15.0027  0.1215  0.1231  0.1231  0.1215  0.1209 
Mixture 3  14.1272  15.2872  14.2790  14.1272  14.1095  0.1037  0.1076  0.1056  0.1037  0.1034 
Mixture 4  17.5804  20.5323  17.8193  17.5804  17.5681  0.2274  0.1985  0.2324  0.2274  0.2274 
Mixture 5  18.6483  19.2550  18.6473  18.6483  18.6134  0.2358  0.2512  0.2349  0.2358  0.2358 
Mixture 6  19.1086  21.3061  19.4464  19.1086  19.0999  0.2598  0.2141  0.2711  0.2598  0.2596 
Mixture 7  15.7100  16.3647  16.4140  16.3326  15.4834  0.1396  0.1389  0.1409  0.1405  0.1364 
Mixture 8  16.4893  17.1885  17.4584  18.2166  16.6785  0.1301  0.1312  0.1326  0.1494  0.1329 
Mixture 9  16.0829  17.2970  17.8173  18.6586  15.9036  0.1591  0.1467  0.1923  0.1870  0.1533 
Mixture 10  16.0358  17.5287  17.9511  18.1627  16.3448  0.1668  0.1624  0.1826  0.1890  0.1685 
IvB Results of Scene II
IvB1 Unmixing results of the spatially uniform mixtures
The FCLS and NCLS were firstly applied to Scene II, with the mean estimates and RMSE results reported in the first and the second columns of Tab. VI. We observe that in this scene both of these algorithms led to high estimation RMSEs, possibly caused by the fact that the intimate interaction of granules resulted in nonlinear mixture effects. Considering this, we then applied the nonlinear unmixing algorithm KHype to the data. The KHype algorithm models the observed pixel by a linear mixture and a nonlinear fluctuation of spectral signatures, and it is a kernel based algorithm to perform the unmixing, see [13] for more details. We used the Gaussian kernel, with . The following methods were also tested linear () [39], MLM [40], Hapke model [24]. The obtained results are reported in the third column of Tab. VI and Tab. VII. We observe that KHype and Hapke model led to reduced RMSEs for most of the mixtures. This showed the usefulness of the nonlinear unmixing.
FCLS  NCLS  KHype  

Red  Green  Blue  White  RMSE  Red  Green  Blue  White  RMSE  Red  Green  Blue  White  RMSE  
Mixture 1  
Mixture 2  
Mixture 3  0.0334  
Mixture 4  
Mixture 5  0.1054  
Mixture 6  0.2909  
Mixture 7  0.0533  
Mixture 8  
Mixture 9  0.0662  
Mixture 10  0.0848  
Mixture 11  
Mixture 12  0.0586  
Mixture 13  0.0867  
Mixture 14 
linear  MLM  Hapke  

Red  Green  Blue  White  RMSE  Red  Green  Blue  White  RMSE  Red  Green  Blue  White  RMSE  
Mixture 1  0.0887  
Mixture 2  0.1181  
Mixture 3  
Mixture 4  0.0798  
Mixture 5  
Mixture 6  0.0905  
Mixture 7  
Mixture 8  0.0878  
Mixture 9  
Mixture 10  
Mixture 11  0.0851  
Mixture 12  
Mixture 13  
Mixture 14  0.0849 
IvB2 Unmixing results of the mixtures with spatial pattern
The true abundance maps of these four mixtures (clipped center square regions) are shown in the first columns in Figs. 10 to 13. The abundance maps estimated using the FCLS, NCLS, and KHype algorithms are shown alongside. We observe that the general spatial patterns of the estimated maps were consistent with those of the groundtruth reference; however, the RMSEs are notable. The KHype algorithm showed better RMSEs than the other two algorithms in mixtures of patterns A, C, and D. A further study on these mixtures is necessary.
IvC Results of Scene III
In this scene, the spectra of the pure printed colors and that of the board are used as the endmembers. Then, the unmixing algorithms were executed to estimate the abundances of the five mixtures. The mean estimates are shown in Tab. VIII. We observe that the board significantly contributed to the spectra because of to its strong reflection. As it was difficult to estimate the contribution from the board, we normalized the sum of the abundances of the printed color to 1 and then compared it with the groundtruth obtained from the layouts of the checkerboards. These RMSEs are shown in Tab. IX. We found that KHype exhibited good performance for the mixture with multiple materials.
FCLS  NCLS  KHype  Bilinear  

Magenta  Yellow  Cyan  Board  Magenta  Yellow  Cyan  Board  Magenta  Yellow  Cyan  Board  Magenta  Yellow  Cyan  Board  
Mixture 1  
Mixture 2  
Mixture 3  
Mixture 4  
Mixture 5  
Mixture 6  
Mixture 7  
Mixture 8 
FCLS  NCLS  KHype  Bilinear  

Mixture 1  
Mixture 2  
Mixture 3  
Mixture 4  
Mixture 5  
Mixture 6  
Mixture 7  
Mixture 8 
V Conclusion
In our work, we created a dataset for evaluating the performance of spectral unmixing algorithms. The dataset provides 3 types of mixture scenes and 36 mixtures that simulate different mixture models. Each pixel consists of 256 spectral bands with a spectral resolution of up to . Both of the endmembers of pure materials and their compositions are priorly known, allowing us to testing unmixing algorithms in a quantitative and an objective manner. Typical unmixing algorithms, including the FCLS, NCLS, KHype, Bilinear, linear, MLM, and Hapke methods were applied to this dataset and led to interpretable results. Unmixing of some complex scenes however requires some further study. In the future, we will on one hand make efforts to enrich the dataset with more settings, and on the other hand, to devise and test unmixing algorithms with these labeled data.
References
 [1] M. Zhao and J. Chen, “A dataset with groundtruth for hyperspectral unmixing,” in Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Valencia, Spain, Jul. 2018, pp. 5081–5084.
 [2] J. M. BioucasDias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, “Hyperspectral unmixing overview: Geometrical, statistical, and sparse regressionbased approaches,” IEEE J. Sel. Top. Appl. Earth Observat. Remote Sens., vol. 5, no. 2, pp. 354–379, 2012.
 [3] J. W. Boardman, “Automating spectral unmixing of aviris data using convex geometry concepts,” JPL, Summaries of the 4th Annual JPL Airborne Geoscience Workshop, vol. 1, pp. 11–14, 1993.

[4]
J. M. Nascimento and J. M. BioucasDias,
“Does independent component analysis play a role in unmixing hyperspectral data?,”
IEEE Trans. Geosci. Remote Sens., vol. 43, no. 1, pp. 175–187, 2005.  [5] M. E. Winter, “Nfindr: An algorithm for fast autonomous spectral endmember determination in hyperspectral data,” in Imaging Spectrometry V. International Society for Optics and Photonics, 1999, vol. 3753, pp. 266–276.
 [6] D. Heize and C. Chang, “Fully constrained least squares linear mixture analysis for material quantification in hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 3, pp. 529–545, 2001.
 [7] L. Miao and H. Qi, “Endmember extraction from highly mixed data using minimum volume constrained nonnegative matrix factorization,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 3, pp. 765–777, 2007.
 [8] M.D. Iordache, J. M. BioucasDias, and Plaza A., “Sparse unmixing of hyperspectral data,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 6, pp. 2014–2039, 2011.
 [9] J. Li, A. Agathos, D. Zaharie, J. M. BioucasDias, Plaza A., and X. Li, “Minimum volume simplex analysis: a fast algorithm for linear hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 9, pp. 5067–5082, 2015.
 [10] M.D. Iordache, J. M. BioucasDias, and A. Plaza, “Total variation spatial regularization for sparse hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 11, pp. 4484–4502, 2012.
 [11] J. Chen, C. Richard, and P. Honeine, “Nonlinear abundance estimation of hyperspectral images with norm spatial regularization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 5, pp. 2654–2665, May. 2014.
 [12] A. Halimi, Y. Altmann, N. Dobigeon, and J. Tourneret, “Nonlinear unmixing of hyperspectral images using a generalized bilinear model,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4153–4162, 2011.
 [13] J. Chen, C. Richard, and P. Honeine, “Nonlinear unmixing of hyperspectral data based on a linearmixture/nonlinearfluctuation model,” IEEE Trans. Signal Process., vol. 61, no. 2, pp. 480–492, 2013.
 [14] Q. Qu, N. Nasrabadi, and T. Tran, “Abundance estimation for bilinear mixture models via joint sparse and lowrank representation,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, pp. 4404–4423, 2014.
 [15] M. Tang, L. Gao, A. Marinoni, P. Gamba, and B. Zhang, “Integrating spatial information in the normalized plinear algorithm for nonlinear hyperspectral unmixing,” IEEE J. Sel. Top. Appl. Earth Observat. Remote Sens., 2017.
 [16] C. A. Bateson, G. P. Asner, and C. A. Wessman, “Endmember bundles: A new approach to incorporating endmember variability into spectral mixture analysis,” IEEE Trans. Geosci. Remote Sens., vol. 38, no. 2, pp. 1083–1094, 2000.
 [17] B. Somers, M. Zortea, A. Plaza, and G. P. Asner, “Automated extraction of imagebased endmember bundles for improved spectral unmixing,” IEEE J. Sel. Top. Appl. Earth Observat. Remote Sens., vol. 5, no. 2, pp. 396–408, 2012.
 [18] M. Xu, L. Zhang, and B. Du, “An imagebased endmember bundle extraction algorithm using both spatial and spectral information,” IEEE J. Sel. Top. Appl. Earth Observat. Remote Sens., vol. 8, no. 6, pp. 2607–2617, 2015.
 [19] N. Keshava and J. F. Mustard, “Spectral unmixing,” IEEE Signal Processing Mag., vol. 19, no. 1, pp. 44–57, 2002.
 [20] O. Eches, N. Dobigeon, C. Mailhes, and J. Tourneret, “Bayesian estimation of linear mixtures using the normal compositional model. application to hyperspectral imagery,” IEEE Trans. Image Process., vol. 19, no. 6, pp. 1403–1413, 2010.
 [21] J. Chen, C. Richard, and P. Honeine, “Estimating abundance fractions of materials in hyperspectral images by fitting a postnonlinear mixing model,” in Proc. IEEE. WHISPERS, 2013, 2013, pp. 1–4.
 [22] Y. Altmann, N. Dobigeon, S. McLaughlin, and J. Tourneret, “Residual component analysis of hyperspectral images: application to joint nonlinear unmixing and nonlinearity detection,” IEEE Trans. Image Process., vol. 23, no. 5, pp. 2148–2158, 2014.
 [23] Y. Altmann, M. Pereyra, and S. McLaughlin, “Bayesian nonlinear hyperspectral unmixing with spatial residual component analysis,” IEEE Trans. CI, vol. 1, no. 3, pp. 174–185, 2015.
 [24] B. Hapke, “Bidirectional reflectance spectroscopy: 1. theory,” J. Geophys. Res., vol. 86, no. B4, pp. 3039–3054, 1981.
 [25] J. Yang, Y.Q. Zhao, J. C.W. Chan, and S. G. Kong, “Coupled sparse denoising and unmixing with lowrank constraint for hyperspectral image,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 1818–1833, 2016.
 [26] J. Sigurdsson, O. Ulfarsson, and J. R. Sveinsson, “Blind hyperspectral unmixing using total variation and sparse regularization,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 11, pp. 6371–6384, 2016.
 [27] P. Giampouras, K. E. Themelis, A. A. Rontogiannis, and K. D. Koutroumbas, “Simultaneously sparse and lowrank abundance matrix estimation for hyperspectral image unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 8, pp. 4775–4789, 2016.
 [28] A. Halimi, J. BioucasDias, N. Dobigeon, G. S. Buller, and S. McLaughlin, “Fast hyperspectral unmixing in presence of nonlinearity or mismodeling effect,” IEEE Trans. Comput. Imag., vol. 3, no. 2, 2017.
 [29] X. Jiang, M. Gong, H. Li, and J. Li, “A twophase multiobjective sparse unmixing approach for hyperspectral data,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 1, pp. 508–523, 2018.
 [30] I. Dopido, M. Zortea, A. Villa, Plaza A., and Gamba P., “Unmixing prior to supervised classification of remotely sensed hyperspectral images,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 4, pp. 760—764, 2011.
 [31] M.D. Iordache, J. M. BioucasDias, and A. Plaza, “Collaborative sparse regression for hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 1, pp. 1–14, 2013.
 [32] N. Akhtar and A. Mian, “RCMF: robust constrained matrix factorization for hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 55, no. 6, pp. 3354–3366, 2017.
 [33] S. Zhang, J. Li, H.C. Li, C. Deng, and A. Plaza, “Spectral–spatial weighted sparse regression for hyperspectral image unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 6, pp. 3265–3276, 2018.
 [34] C. Shi and L. Wang, “Linear spatial spectral mixture model,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 6, pp. 3599–3611, 2016.
 [35] Y. Qian, S. Jia, J. Zhou, and A. RoblesKelly, “Hyperspectral unmixing via sparsityconstrained nonnegative matrix factorization,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4282–4297, 2011.
 [36] X. Lu, H. Wu, Y. Yuan, P. Yan, and X. Li, “Manifold regularized sparse nmf for hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 51, no. 5, pp. 2815–2826, 2013.
 [37] S. Serranti, R. Palmieri, and G. Bonifazi, “Hyperspectral imaging applied to demolition waste recycling: innovative approach for product quality control,” J. Electron. Imaging, vol. 24, no. 4, pp. 043003, 2015.
 [38] A. Zare and P. Gader, “Sparsity promoting iterated constrained endmember detection in hyperspectral imagery,” IEEE Geosci. Remote Sens. Lett., vol. 4, no. 3, pp. 446–450, 2007.
 [39] A. Marinoni and P. Gamba, “A novel approach for efficient linear hyperspectral unmixing,” IEEE J. Sel. Top. Sign. Proces., vol. 9, no. 6, pp. 1156–1168, 2015.
 [40] R. Heylen and P. Scheunders, “A multilinear mixing model for nonlinear spectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 1, pp. 240–251, 2016.
Comments
There are no comments yet.