Multi-scale discriminant saliency with wavelet-based Hidden Markov Tree modelling

Bottom-up saliency, an early stage of human visual attention, can be considered as a binary classification problem between centre and surround classes. Discriminant power of features for the classification is measured as mutual information between distributions of image features and corresponding classes . As the estimated discrepancy very much depends on considered scale level, multi-scale structure and discriminant power are integrated by employing discrete wavelet features and Hidden Markov Tree (HMT). With wavelet coefficients and Hidden Markov Tree parameters, quad-tree like label structures are constructed and utilized in maximum a posterior probability (MAP) of hidden class variables at corresponding dyadic sub-squares. A saliency value for each square block at each scale level is computed with discriminant power principle. Finally, across multiple scales is integrated the final saliency map by an information maximization rule. Both standard quantitative tools such as NSS, LCC, AUC and qualitative assessments are used for evaluating the proposed multi-scale discriminant saliency (MDIS) against the well-know information based approach AIM on its released image collection with eyetracking data. Simulation results are presented and analysed to verify the validity of MDIS as well as point out its limitation for further research direction. Preprint submitted to Elsevier February 1, 2013 ar X iv :1 30 1. 76 41 v1 [ cs .C V ] 3 1 Ja n 20 13 1. Visual Attention Computational Approach Visual attention is a psychological phenomenon in which human visual systems are optimized for capturing scenic information. Robustness and efficiency of biological devices, the eyes and their control systems, visual paths in the brain have amazed scientists and engineers for centuries. From Neisser [1] to Marr [2], researchers have put intensive effort in discovering attention principles and engineering artificial systems with equivalent capability. For decades, this research field has been dominated by visual attention principle , proposing an existence of a saliency map for attention guidance. The idea is further promoted in Feature Integration Theory (FIT) [3] which elaborates computational principles of saliency map generation with centre-surround operators and basic image features such as intensity, orientation and colour. Then, Itti et al. [4] implemented and released the first complete computer algorithms of FIT theory . Feature Integration Theory is widely accepted as principles behind visual attention partly due to its utilization of basic image features such as colour, intensity, and orientation. Moreover, this hypothesis is supported by several evidences from psychological experiments. However, it only defines theoretical aspects of saliency maps and visual attention , but does not investigate how such principles would be implemented algorithmically. This lack of implementation details leaves research field open for many later saliency algorithms [4, 5, 6, 7], etc. Saliency might be computed as a linear contrast between features of central and surrounding environments across multiple scales by centre-surround operators. Saliency is also modelled as phase difference in Fourier Transform Domain [8], or its value depends on statistical modelling of the local feature distribution [6]. Though many approaches are mentioned in the long and rich literature of visual saliency, only a few are built on a solid theory or linked to other well-established computational theory. Among the approaches, Neil Bruce’s work [9] nicely established a bridge between visual saliency and information theory. It puts a first step for bridging two alien fields; moreover, visual attention for first time could be viewed as information system. Then, information based visual saliency has continuously been investigated and developed in several works [10, 11, 12, 13]. The distinguish points between these works are computational approaches for retrieving information from features. The process attracts much interhttp://ilab.usc.edu/toolkit/