Optimal Eigenfeature Selection by Optimal Image Registration

Large collections of images can be indexed by projections on a few “eigenfeatures”, the dominant eigenvectors of the images covariance matrix. A preliminary step of registering the images is common practice. A quantitative analysis of what is being gained by registration was not performed in previous work, and heuristics were used to determine on what to register the images. We show that the registration improves the accuracy of indexing, and optimal improvement is obtained when the images are registered on their eigenfeatures subspace. Similarly, if multiple images are to be registered on a subspace, the optimal subspace is the one spanned by the dominant eigenfeatures of the registered images. An algorithm that simultaneously registers the images and computes their eigenfeatures is proposed. The key idea is to iterate the following two steps: 1. Eigenfeatures are computed from the images. 2. New images are computed by registering the images on the subspace of these eigenfeatures. In the next iteration, Step 1 is applied to the set of images that were most recently computed in Step 2. It is demonstrated that the algorithm produces improved eigenfeatures and registers multiple images.