Pyramidal Implementation of the AÆne Lucas Kanade Feature Tracker Description of the algorithm

1 Problem Statement Let I and J be two 2D grayscaled images. The two quantities I(x) = I(x; y) and J(x) = J(x; y) are then the grayscale values of the two images at the location x = [x y] , where x and y are the two pixel coordinates of a generic image point x. The image I will sometimes be referenced as the rst image, and the image J as the second image. For practical issues, the images I and J are discret function (or arrays), and the upper left corner pixel coordinate vector is [0 0] . Let nx and ny be the width and height of the two images. Then the lower right pixel coordinate vector is [nx 1 ny 1] T . Consider an image point u = [ux uy] T on the rst image I . The goal of feature tracking is to nd the location v = [ux + dx uy + dy] T on the second image J such as I(u) and J(v) are \similar". The vector d = [dx dy] T is the image velocity at u, also known as the optical ow at u. In addition to a translation component d, let us assume that the image undergoes an aÆne deformation between I and J in the vicinity of the two image feature points u and v. Following this assumption, let us introduce the aÆne transformation matrix A: