CSE 190A Research Proposal: Super-Resolution

Super-resolution seeks to create a high-quality image of a scene from several lowquality images of the same scene. Though a literature of super-resolution applications exists, implementing super resolution successfully is apparently quite difficult. In fact, a student unsuccessfully attempted to implement super-resolution in a previous offering of this course. I will attempt the same task, but using a different approach that should yield better compartmentalization of bugs and inaccuricies than was enjoyed by the unsuccessful student. 1 Project description Multiple-image super-resolution seeks to combine the information from several low-quality photographs of a scene to create a single high-quality picture. The field is mature enough that there exists an iPhone application, ClearCam by Occipital, that takes 6 images in quick succession, and then uses them to perform super resolution with about 40 seconds of processing on the iPhone’s processor. Another approach to super resolution uses a single image and prior information about the scene to guess the values of missing pixels. This approach is called “hallucination”, and will not be addressed in this project. In multiple-image super-resolution, the low quality pictures may be of low resolution, be subject to blurring caused by poor focus, and have noisy pixel values. David Capel’s thesis [2] and a follow-up paper [1] describe a framework for performing super-resolution. In the first step, image registration, corresponding points in the low quality images are identified. This algorithm is described concisely in Figure 4 of [1]. These corresponding points are used to compute geometric and photometric transformations between the images. Here, a geometric transformation accounts for a change in the point of view of the camera, and a photometric transformation accounts for changes in illumination and exposure between shots. Once the corresponding points are identified, the main step in super-resolution is performed. The pixel values from the low quality images are represented as the results of a process that creates low quality images from image scenes, and inference is performed to find the ML or MAP value of pixels in the image scene given the low quality pixels. In the winter 2006 offering of CSE 190A, Jesse Cirimele attempted to implement super-resolution [3]. He first implemented image registration as a prerequisite to performing super-resolution on real images, and appears not to have succeeded in implementing super-resolution by the end of the quarter. Given Jesse’s lack of success, and the general difficulty of the problem as described to me by Serge Belongie, I plan to start with a simple subproblem of super-resolution before tackling the full problem. In particular, I plan to start with synthetic images, the corresponding points of which I control, so that I may test my super-resolution implementation independent of a potentially unreliable image registration subroutine. Once this simpler problem is solved, I will extend the implementation to deal with real images.