Plenoptic Models in Robot Vision

Plenoptic models, representatives are the lightfield or the lumigraph, have been successfully applied in computer vision and computer graphics in the past five years. The key idea is to model objects and scenes using images and some extra information like camera parameters or coarse geometry. The model differs from CAD–models in the photorealism that can be achieved and is thus superior in applications where the realism of the model is of importance. From the point of view of learning based robot vision these models have the additional advantage that they can be acquired fully automatically from image sequences. The paper shows how such models can be applied in robot vision. Starting with the theoretical principles, the automatic generation of plenoptic models is discussed. An new method is introduced for direct rendering from arbitrarily taken real views. The adaptive use of geometric information makes it possible to scale the model accuracy with respect to available computation time. In two typical applications from robot vision the benefits of this kind of model is demonstrated. The experimental results proof our claim that plenoptic models are useful in robot vision.