High Quality Novel View Rendering from Multiple Cameras

The research presented in this thesis is targeted towards obtaining high quality novel views of a dynamic scene using video from multiple wide-baseline views, with free-viewpoint video as the main application goal. The research has led to several novel contributions to the 3D reconstruction computer vision literature. The first novel contribution of this work is the exact view-dependent visual hull, a method to efficiently reconstruct a three dimensional representation of the scene with respect to a given viewpoint. This approach includes two novel contributions which allow the reconstruction to be performed in the image domain. The first is the Visual Hull Visible Intersection Theorem, an efficient way to identify points on the visual hull surface from the input images. The second is the use of the cross ratio to globally order intersections from individual images, avoiding the need for explicit 3D reconstruction of every point. This not only increases the efficiency of the reconstruction, it also produces an exact representation of the visual hull by maintaining pixel accuracy in the original images. A method for for producing high quality novel views through efficient local surface refinement is introduced. This reduces artefacts such as ghosting from incorrect correspondence between views when using the visual hull. A representation for rendering the refined surfaces in real-time with a user-controllable viewpoint is introduced. An alternative method for producing high quality novel views from wide-baseline cameras using a global optimisation to refine the entire surface is presented. The goal of this is to produce a continuous surface which removes depth artefacts and represents the overall shape of the scene. The optimisation is constrained by surface contours called rims extracted from the visual hull, to avoid overrefinement of the surface. The final novel contribution of this thesis is the safe hull, the first visual hull based reconstruction method which guarantees production of a surface without phantom volumes (an artefact of visual hull reconstruction, due to multiple objects in a scene). The safe hull identifies volumes inside the visual hull which only contain foreground i.e. the object to be reconstructed. This approach uses a novel geometric constraint, utilising information gained from the exact viewdependent visual hull, unlike other solutions which are either heuristic or require additional cameras.