Towards Optimal Non-rigid Surface Tracking

This paper addresses the problem of optimal alignment of non-rigid surfaces from multi-view video observations to obtain a temporally consistent representation. Conventional non-rigid surface tracking performs frame-to-frame alignment which is subject to the accumulation of errors resulting in a drift over time. Recently, non-sequential tracking approaches have been introduced which re-order the input data based on a dissimilarity measure. One or more input sequences are represented in a tree with reduced alignment path length. They demonstrate a reduced drift and increased robustness to large non-rigid deformations. However, jumps may occur in the aligned mesh sequence where branches of the tree meet due to different error accumulation along them. Optimisation of the tree for non-sequential tracking, which minimises the errors in temporal consistency due to both the drift and the jumps, is proposed. A novel cluster tree enforces sequential tracking in local segments of the sequence while allowing global non-sequential traversal among these segments. This provides a mechanism to create a tree structure which reduces the number and size of jumps between branches and limits the lenght of branches as well. Comprehensive evaluation is performed on a variety of challenging non-rigid surfaces including face, cloth and people. It demonstrates that the proposed cluster tree achieves better temporal consistency than the previous sequential and non-sequential tracking approaches. Quantitative analysis on a created synthetic facial performance also shows an improvement by the cluster tree.