In this paper, we propose a new video genre detection using semantic classification with multi-modal features. MPEG-7 audio-visual descriptors are used as multi-modal features. From the low-level multimodal features, genre as high-level semantic meaning is detected by using GINI index in Classification And Regression Tree (CART) algorithm. Experimental results show that the proposed method is u...

Medical image registration is time-consuming but can be sped up employing parallel processing on the GPU. Normalized mutual information (NMI) is a well performing similarity measure for performing multi-modal registration. We present CUDA based solutions for computing NMI on the GPU and compare the results obtained by rigidly registering multi-modal data sets with a CPU based implementation. Ou...

Sensitive and selective sensor system is required to detect causes of disease. It is able to detect the cause of the disease more effectively by using multi-modal properties of nano-bio sensors. Some peak detection algorithms which are suitable for implementing with micro-controller are compared to apply to multi-modal nano-bio sensors in this paper. It is shown that autothreshold peak detectio...

We address the estimation of the interest and comprehension level of an audience in poster sessions. Compared to lecture presentations, the audience’s behaviors such as gazing and backchannels are more observable in poster presentations. These multi-modal behaviors are presumably related with their interest and comprehension level. We also assume that the interest and comprehension level can be...

WE PROPOSE A NEW FRAMEWORK TO UNDERSTAND singing accuracy, based on multi-modal imagery associations: the MMIA model. This model is based on recent data suggesting a link between auditory imagery and singing accuracy, evidence for a link between imagery and the functioning of internal models for sensorimotor associations, and the use of imagery in singing pedagogy. By this account, imagery invo...

The rapid increase in multimedia data transmission over the Internet necessitates the multi-modal summarization (MMS) from collections of text, image, audio and video. In this work, we propose an extractive multi-modal summarization method that can automatically generate a textual summary given a set of documents, images, audios and videos related to a specific topic. The key idea is to bridge ...

The contribution of this paper is to compare paradigms coming from the classes of parametric, and non-parametric techniques to solve the decision fusion problem encountered in the design of a multi-modal biometrical identity verification system. The multi-modal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, c...

The benefits of a network of cooperative multi-modal sensors are presented in the contexts of three different human-computer interaction systems. First, a robust automatic teleconferencing system with multiple microphones and cameras is used to illustrate the benefits of multi-modal sensor fusion. The integrated speaker localization system doubles the original accuracy of the individual sensors...

The car of the future will have many new information sources— including telematics systems, navigation systems and Advanced Driver Assistance Systems (ADAS)—that will compete for a driver’s limited cognitive attention. If they are implemented as completely separate systems then cognitive overload and driver distraction are inevitable outcomes. However, if they are implemented as an integrated i...

RGBD sensors, such as the Microsoft Xbox Kinect [1] are types of multi-modal perceptual sensors that have appeared in recent years. RGBD sensors have become standard perceptual tools for robots as they provide a unique multi-modal approach to perception. A vital pre-cursing challenge in object grasping and manipulation is object pose recognition. A robot must identify the pose (i.e. orientation...