Incremental recognition of multi-object behaviour using hierarchical probabilistic models

In this contribution we present a new methodological framework and first results for real-time monitoring of object behaviour in aircraft servicing scenes, such as arrival preparation, unloading, tanking and others, based on video streams from several cameras. The focus is on incremental real-time interpretation of multiple object tracks. We show that the temporal structure of complex, partially coordinated object behaviour such as aircraft servicing can be modelled by a Bayesian Compositional Hierarchy (BCH). This is a recently developed kind of Bayesian Network where aggregates are modelled with unrestricted distributions, whereas the dependency structure between aggregates is restricted to correspond to the tree structure of the compositional hierarchy. This allows efficient updating when evidence is incorporated incrementally. For the domain of service operations, a BCH has been constructed for modelling the durations of activities and delays between them. The BCH is primarily used to provide a ranking of alternative partial interpretations and control the interpretation process according to the beam search paradigm. In addition, a BCH can provide estimates of missing data based on current evidence, for example, regarding the duration of a servicing operation. We explain the structure of aggregates constituting the aircraft servicing BCH and demonstrate evidence-based updates as well as predictions.