Semantic filtering of video content

Semantic filtering of multimedia content is a challenging problem. The gap that exists between low-level media features and high-level semantics of multimedia is difficult to bridge. We propose a flexible probabilistic graphical framework to bridge this gap to some extent and perform automatic detection of semantic concepts. Using probabilistic multimedia objects (multijects) and a network of such objects (multinet) we support semantic filtering. Discovering the relationships that exist between semantic concepts, we show how the detection performance can be improved upon. We show that concepts which may not be directly observed in terms of media features, can be inferred based on their relation with those that are already detected. Heterogeneous features also can be fused in the multinet. We demonstrate this by inferring the concept outdoor based on the five detected multijects sky, snow, rocks, water and forestry and a frame-level global-features based outdoor detector.