Smoothness in Video Coding

Most motion compensation methods dealt with in the literature make strong assumptions about the smoothness of the underlying motion eld. For instance, block-matching algorithms assume a blockwise-constant motion eld and are adequate under translational motion models; triangle motion compensation assumes a piecewise-planar motion eld, and this model captures zooming, warping and rotational motion very well. Time-varying imagery however, often contains both types of motion (as well as others) and hence exhibits a high degree of spatial variability in terms of its motion eld smoothness properties. We develop a simple method that performs spatial adaptation of the smoothness of the motion eld and demonstrate substantial improvements in video quality over a wide range of bit rates. To this end, we introduce the notion of a motion eld that is characterized by a set of labels which permits one to adaptively switch among a collection of diierent motion models locally. Each model is characterized by the smoothness of the underlying motion eld. The switched framework for motion compensation performs signiicantly better than each of its constituent motion models, in terms of both visual quality and signal-to-noise ratio.