Probabilistic Logical Sequence Learning for Video

Understanding complex, dynamic scenes of real-world activities from low-level sensor data is of central importance for intelligent systems. The main difficulty lies in the fact that complex scenes are best described in high-level, logical formalisms, while sensor data usually consists of many low-level features. We first propose a method to obtain a logical representation of real-world, dynamic scenes based on input video stream solely. We focus on representing the video data using probabilistic relational sequences as a natural way to incorporate sensor uncertainty. They allow us to work with structured terms, and in addition they capture the inherent uncertainty of object detection. Further on, we employ r-grams as the probabilistic logical learning model for this application. In a first step we use r-grams in a simple setting and we show their viability in card games. We also show how r-grams can be upgraded to deal with uncertain observations.