Effective retrieval and visual analysis in multimedia databases

Based on advances in acquisition, storage, and dissemination technology, increasing amounts of multimedia content such as images, audio, video, or 3D models, become available. The Feature Vector (FV) paradigm is one of the most popular approaches for managing multimedia content due to its simplicity and generality. It maps multimedia elements from object space to metric space, allowing to infer object similarity relationships from distances in metric space. The distances in turn are used to implement similarity-based multimedia applications. For a given multimedia data type, many different FV mappings are possible, and the effectiveness of a FV mapping can be understood as the degree of resemblance of object space similarity relationships by distances in metric space. The effectiveness of the FV mapping is essential for any application based on it. Two main ideas motivate this thesis. We first recognize that the FV approach is promising, but needs attention of FV selection and engineering in order to serve as a basis for building effective multimedia applications. Secondly, we believe that visualization can contribute to building powerful user interfaces for analysis of the FV as well as the object space. This thesis focuses on supporting a number of important user tasks in FV-based multimedia databases. Specifically, we propose innovative methods for (a) effective processing of content-based similarity queries, (b) FV space visualization for discrimination analysis, and (c) visualization layout generation for content presentation. The methods are applied and evaluated on a number of specific multimedia data types such as 3D models, images, and time series data, and are expected to be useful in many other multimedia domains. Effective retrieval in 3D databases (Chapter 2). We review and classify a significant number of recently proposed FV extractors supporting the 3D model domain. Extensive effectiveness evaluation experiments are performed for many FV extractors on a number of benchmarks. Methods for improving retrieval effectiveness by forming static and query-dependent combinations of FVs are researched. Experiments show significant improvements in retrieval precision (quality of the answer sets) to be achievable. Visual FV space analysis (Chapter 3). We explore the usage of interactive 2D projections for retrieval and organization of multimedia content in a multi-FV 3D retrieval system. Selforganizing maps (SOMs) have shown to be appropriate to this end. We propose a PCA-based visualization method for supervised visual discrimination analysis in FV space. Also, SOMbased techniques are explored for unsupervised estimation of FV space discrimination power. Both visualizations can be used for addressing the FV selection problem, and for fine tuning FV-based multimedia applications.