Image Based Quantitative Mosaic Evaluation with Artificial Video

Interest towards image mosaicing has existed since the dawn of photography. Many automatic digital mosaicing methods have been developed, but unfortunately their evaluation has been only qualitative. Lack of generally approved measures and standard test data sets impedes comparison of the works by different research groups. For scientific evaluation, mosaic quality should be quantitatively measured, and standard protocols established. In this paper two methods are proposed for creating video images for testing planar mosaicing performance. The first method produces artificial videos using a virtual camera and allows to evaluate also superresolution mosaicing. The second method uses a real camera to take pictures of a marked surface. More importantly, a measuring method is presented for evaluating the performance of different mosaicing algorithms, and the method is showcased with the root-mean-squared error metric. Important evaluation issues are addressed, especially mosaic coverage. Three artificial test videos are ran through a real-time mosaicing system to demonstrate the proposed mosaic quality evaluation framework, which consists of the method for generating artificial video and the measuring method.