Evaluating the success of prediction and retrieval systems depends upon a reliable reference standard, a ground truth. The ideal gold standard is expected to result from the marking, labeling, and rating of domain experts of the image of interest. However experts often disagree and this lack of agreement challenges the development and evaluation of image-based feature prediction of expert-defin...

Two significant problems in content based retrieval methods are (1) Accuracy: most of the current content based image retrieval methods have not been quantitatively compared nor benchmarked with respect to accuracy and (2) Efficiency: image database search methods must be analyzed for their computational efficiency and interrelationships. We assert that the accuracy problem is due to the genera...

The aim of this study was to increase understanding of the psychology of deception by mapping the reasoning of guilty and innocent mock suspects who deny a transgression. Based on previous research, we proposed that suspects will engage in two major forms of regulation: impression management, which requires the purposeful control of nonverbal and demeanor cues; and information management which ...

The manual curation of knowledge bases is a bottleneck in fast paced domains where new concepts constantly emerge. Identification of nascent concepts is important for improving early entity linking, content interpretation, and recommendation of new content in real-time applications. We present an unsupervised method for generating pseudo-ground truth for training a named entity recognizer to sp...

Content-based image retrieval (CBIR) addresses the problem of assisting a user to retrieve images from unannotated databases, based on features that can be automatically derived from the images. Today, there exists several CBIR systems based on different methods. Only few attemps to benchmark these have been made, although the usefulness of benchmarking is undeniable in the development of diffe...

The performance of a Content-Based Image Retrieval (CBIR) system presented in the form of Precision-Recall or PrecisionScope graphs offers an incomplete overview of the system under study: the influence of the irrelevant items is obscured. In this paper, we propose a comprehensive and well normalized description of the ranking performance compared to the performance of an Ideal Retrieval System...

Evaluating the success of prediction and retrieval systems depends upon a reliable reference standard, a ground truth. The ideal gold standard is expected to result from the marking, labeling, and rating of domain experts. However experts disagree and lack of agreement challenges the development and evaluation of image-based feature prediction. This paper addresses the success and limitations i...

In this paper, we address an important yet controversial issue in content-based image retrieval (CBIR): performance evaluation (PE). The goal is to develop an objective and comprehensive protocol for quantitative evaluation and comparison of CBIR performances among different approaches. The protocol consists of two elements: acquisition of appropriate ground truth data, and definition of quanti...

In the research area of automatic web information extraction, there is a need for permanent and annotated web page collections enabling objective performance evaluation of different algorithms. Currently, researchers are suffering from the absence of such representative and contemporary test collections, especially on web tables. At the same time, creating your own sharable web page collections...

Text detection and tracking is an important step in a video content analysis system as it brings important semantic clues which is a vital supplemental source of index information. While there has been a significant amount of research done on video text detection and tracking, there are very few works on performance evaluation of such systems. Evaluations of this nature have not been attempted ...