Parameterisation and Probability in Image Alignment

In this paper we investigate extending the gradient-based inverse compositional image alignment method described by Baker and Matthews [1] by formulating the alignment process probabilistically using Bayes rule to obtain a posterior estimate of the alignment. The probabilistic formulation makes use of prior statistical information on the aligning function parameters, and we investigate the use of arbitrary parameterisation of this function to match the physical system generating the warp. We compare the probabilistic method to the standard inverse compositional algorithm by using affine alignment to track locally planar image regions through image sequences in real-time. We show that the stabilising effect of probabilistic alignment gives more reliable results with little effect on alignment speed. For this application we also find that the choice of warp parameterisation is significant in its own right, as we get much better results from the standard inverse compositional algorithm with a more physically motivated affine parameterisation. 1. BACKGROUND TO IMAGE ALIGNMENT Image alignment as described here is the process of bringing into alignment two image regions by finding the parameters of a known function relating position in one region to position in the other, where the parameters are initially only approximately known. Alignment is useful for any application where pixels in a local area are expected to move in way which can be modelled by a function of position; for example, for tracking rigid objects of known surface shape moving in 3D, tracking the images from a rotating camera, or tracking non-rigid objects which deform in a way which can realistically be modelled. The image alignment problem has been studied for a number of years. Early work using a gradient method was done by Lucas and Kanade [2], and many authors have since looked at modifications to this [3][4][5][1][6]. There are also a range of methods which perform image alignment in very different ways for different applications, for example, optic flow [7], look up techniques [8], or feature matching [9] 1.1. Inverse compositional image alignment The work described here is based on the inverse compositional image alignment algorithm of Baker and Matthews [1], which is a more efficient formulation of the Lucas and Kanade method. Using Baker and Matthews’ notation, the method aims to align an image with a template image , as illustrated in Figure 1. , and represent the intensity in images and at image position . The alignment function maps an image position to a new position , and is a function of the image position and a set of warp parameters . The method aims to minimise the following cost function with respect to : ! "$# (1) where corresponds to the current estimate of the set of warp parameters needed to bring the two images into alignment, and variation of is used to obtain the most optimal alignment from the image data. The summation is done on a set of pixels in the region of interest. This might be all the pixels in an area, or the subset of these for which the image intensity gradient is significant. The warp estimate is then updated as follows: