Towards a Fusion of Region-based and Saliency-based Models

This thesis addresses the problems of automatic image annotation (AIA) for the purpose of image indexing & retrieval in an Annotation Based Image Retrieval (ABIR) system. Specifically, we study different models of image representation in the AIA area. Up to our knowledge, nobody has tried to combine the following approaches for image representation: region-based approach and saliency-based approach. We think this combination will give a model which captures at the same time the global information and the details of objects. The proposed approach is composed of three main stages. In the first part, the image processing stage consists of building an image representation and extracting the visual features from the image entities. Image presentation is driven by the segmentation process and the keypoint detection algorithm. Each region or keypoint is associated with a set of low-level features (color, histogram, texture, spatial location and invariant local features). The second stage consists of leaning the relationship between semantics and visual features in image. Machine learning algorithm has been used to cluster the visual features into visterms which is an intermediate representation between high-level semantics and low-level features. The learning phase results in a co-occurrence matrix of words and visterms. The fusion of different models is expressed by the fusion of its corresponding word-by-term matrix. The high dimension of the resulting matrix makes the matching process more expensive. This can be reduced by using dimensionality reduction methods (i.e. Latent Semantic Analysis). The last stage consists of the automatic annotation propagation scheme for a new image. The latter will be quantized in term of the visterm frequency. The propagated words list is then ranked based on the cosine similarity between visterms extracted in the image and visterms associated with words. Experiments are conducted on Corel image datasets containing 5000 images and show good annotation performance and demonstrate the improvement of the fusion models compared to the Translation model. keywords: Automatic Image Annotation, Machine Learning, Image Processing.