Urban Vegetation Recognition Based on the Decision Level Fusion of Hyperspectral and Lidar Data

Introduction: Information about vegetation cover and their health has always been interesting to ecologists due to its importance in terms of habitat, energy production and other important characteristics of plants on the earth planet. Nowadays, developments in remote sensing technologies caused more remotely sensed data accessible to researchers. The combination of these data improves the object classification and recognition results. Recently, hyperspectral and Lidar data has been used for vegetation covers classification. The spectral information derived from hyperspectral data is used to classify and identify the vegetation cover. However; due to the spectral similarities between various vegetation types, false positive results are increased. Using relief information extracted from Lidar data can solve these kinds of errors and can be very efficient for improving the object recognition results. Spectral similarities and spatial adjacencies between various kinds of objects, shadow and occluded areas behind high rise objects as well as the complex relationships between various object types lead to the difficulties and ambiguities in vegetation recognition among other objects in urban areas. Therefore, new procedures and higher levels of modifications should be considered for improving the object recognition results. In recent years, the multi-agent systems have been considered as one of the most powerful tools for solving the problems of automatic object recognition in urban areas. Method: According to the difficulties of vegetation recognition in complex urban areas, the proposed object recognition in this paper is a decision level fusion strategy between hyperspectral and Lidar data based on the capabilities of the multi-agent systems. Vegetation indices from hyperspectral image are used as spectral features in the knowledge base. Moreover, digital surface model which is produced from Lidar data is used for height features extraction. After producing a rich knowledge base containing the spectral and height based features, the proposed hierarchical classification is performed which is composed of two steps; step 1: initial vegetation candidate recognition, step 2: vegetation classification based on the capabilities of the multi-agent systems. Applying the optimum thresholds on the normalized difference vegetation index in the first step produces a binary image containing the initial vegetation candidates. The multi-agent system in the second step of the proposed method in this paper contains several object recognition agents (one agent per each vegetation cover type), a coordinator agent and a yellow page. The object recognition agents have three layered internal architecture and use the belief-desire-intention (BDI) reasoning model.  Results: The capabilities of the proposed multi-agent vegetation recognition algorithm in this paper is evaluated based on the hyperspectral and Lidar data collected from the University of Houston and the surrounding areas. Four object recognition agents are defined for trees, healthy grass, water-stress grass and artificial grass. These four object recognition agents perform their reasoning based on the pre-defined spectral and height features in the knowledge base. The obtained results indicate the overall accuracy of about 87% from the proposed multi-agent hyperspectral and Lidar decision fusion strategy. The obtained results from performing the same multi-agent system only on the hyperspectral image (without considering Lidar data) have the overall accuracy for about 71%.