Validation of Abundance Map Reference Data for Spectral Unmixing

The purpose of this study is to validate the accuracy of abundance map reference data (AMRD) for three airborne imaging spectrometer (IS) scenes. AMRD refers to reference data maps (“ground truth”) that are specifically designed to quantitatively assess the performance of spectral unmixing algorithms. While classification algorithms typically label whole pixels as belonging to certain ground cover classes, spectral unmixing allows pixels to be composed of fractions or abundances of each class. The AMRD validated in this paper were generated using our previously-proposed remotely-sensed reference data (RSRD) technique, which spatially aggregates the results of standard classification or unmixing algorithms from fine spatial-scale IS data to produce AMRD for co-located coarse-scale IS data. Validation of the three scenes was accomplished by estimating AMRD in 51 randomly-selected 10 m×10 m plots, using seven independent methods and observers. These independent estimates included field surveys by two observers, imagery analysis by two observers and RSRD by three algorithms. Results indicated statistically-significant differences between all versions of AMRD. Even AMRD from our two field surveys were significantly different for two of the four ground cover classes. These results suggest that all forms of reference data require validation prior to use in assessing the performance of classification and/or unmixing algorithms. Given the significant differences between the independent versions of AMRD, we propose that the mean of all (MOA) versions of reference data for each plot and class is most likely to represent true abundances. Our independent versions of AMRD were compared to MOA to characterize error and uncertainty. Best case results were achieved by a version of imagery analysis, which had mean coverage area differences of 2.0%, with a standard deviation of 5.6%. One of the RSRD algorithms was nearly as accurate, achieving mean differences of 3.0%, with a standard deviation of 6.3%. Further analysis of statistical equivalence yielded an overall zone of equivalence between [−7.0%, 7.2%] for this version of RSRD. The relative accuracy of RSRD methods is promising, given their potential to efficiently generate scene-wide AMRD. These results provide the first known validated abundance level reference data for airborne IS data.