Near Set Index in an Objective Image Segmentation Evaluation Framework

The problem considered in this article∗is how to evaluate image segmentations objectively. An image segmentation is the partition of an image into its constituent parts. Until recently, the evaluation of image segmentations has been largely subjective. Much work has been done in developing metrics for evaluating segmentations with respect to ground truth images. However, little has been done in terms of evaluation without an “expert.” An objective technique for evaluating image segmentations with respect to ground truth images called the Normalized Probabilistic Rand (NPR) index has been proposed (Unnikrishnan et al., 2007). This method evaluates a segmented image by way of soft non-uniform weighting of pixel pairs between the segmented images and one or more ground truth images. The NPR index works well in comparing and benchmarking image segmentation algorithms. However, it is not reasonable to assume that online learning image classification systems will always have access to ground truth images. In this article, we propose an objective metric based on near sets (Henry and Peters, 2007, Peters, 2007b) and information content (MacKay, 2003) of a segmentation called the Near Set Index (NSI) for evaluating image segmentations that does not depend on comparison with ground truth images. Information content provides a measure of the variability of pixel intensity levels within an image and takes on values in the interval [0, log2 L] where L is the number of grey levels in the image. Near set theory provides a framework for representation of groups of similar pixel windows. Within this framework pixel windows can be approximated and degrees of similarity or nearness can be measured. This is advantageous both because it closely parallels our current views on human perception of complex scenes (Fahle and Poggio, 2002) and is tightly coupled to the preprocessing (i.e., feature extraction) stage of object recognition systems (Duda et al., 2001). The availability of an objective segmentation evaluation measure facilitates object recognition, computer vision, or machine learning used to solve image classification problems. The contribution of this article is the introduction of a Near Set Index (NSI) that provides a basis for automation of an objective segmentation evaluation method that is not dependent on ground truth images. Further, a comparison of the results using the NSI and NPR indices is given.