Pose Estimation for Sensors Which Capture Cylindric Panoramas

This paper shows that there exist linear models for sensor pose estimation for multi-view panoramas defined by a symmetric or leveled pair of cylindric images. It assumes that pairs of corresponding points have been detected already in those pairs of panoramas. For the first time a cost function is formulated whose minimization solves the pose estimation problem for these two general cases of multi-view panoramas, specified by unconstrained sensor parameter values but only minor constraints on sensor poses. (Note that due to the non-linearity of the panoramic projection geometry, the modeling of sensor pose estimation typically results into non-linear forms which incur numerical instability.) 1 Review and Basic Notions A panoramic image is recorded by a panoramic sensor, such as a rotating camera. Sensor pose estimation deals with recovering the relative pose of two (calibrated) sensors. Compared to planar images or catadioptric images, there is very few literature on sensor pose estimation from cylindric panoramas. Ishikuro el at. [1] dealt with a very restricted case of the sensor pose estimation problem, in which the given panoramas are acquired at the same altitude and with parallel rotation axes. Kang and Szeliski [2] discussed the sensor pose estimation problem only for single-center panoramas. Neither generalized to the multi-view case (i.e., different intrinsic sensor parameters and arbitrary sensor poses) nor practically relevant cases (e.g., multi-view panoramas as in [3, 4]) of sensor pose estimation have been studied or discussed in the literatures before. This paper provides (for the first time) a cost function whose minimization solves the pose estimation problem for two general cases of cylindric panoramas. A 360◦ cylindric panoramic image can be acquired by various means, such as a rotating video or matrix-sensor camera, a catadioptric sensor (with a subsequent mapping onto a cylinder), or a rotating sensor-line camera, as commercially available from various producers since the late 1990s. For simplifying our discussion, we assume a sensor model close to the latter one which has a fixed rotation axis and takes images consecutively at equidistant angles. (Rotating 2 Fay Huang, Reinhard Klette, and Yun-Hao Xie Fig. 1. Sensor model for cylindric panoramas, showing three image columns with their projection centers. sensor-line cameras allow maximum accuracy, and have been used, e.g., in major architectural photogrammetric projects; see [5]). The projection center of the camera does not have to be on the rotation axis. In the case of an off-axis distance R > 0, the resulting panoramic images are refereed to as multi-projection center panoramas. A multi-view panorama [6] is a set of multior single-projection center cylindric panoramas which were recorded at different locations or with different capturing parameters. In particular, they might be acquired with respect to different rotation axes. In comparison to a single axial panorama [7, 8], the advantages of multi-view panoramas are known to include enlarged visibility and improved stereo reconstruction opportunities; in short, they define multi-view image analysis for the cylindric panoramic case. 2 Cylindric Panoramas General Case We generalize from various panoramic imaging models [1, 7, 9]. The model consists of multiple projection centers and a cylindric image surface; see Fig. 1. C denotes a projection center. Projection centers are uniformly distributed on the base circle which is in the base plane and has center O, which is also the origin of the sensor coordinate system. The off-axis distance R (radius of the base circle) describes the distance between any projection center and the rotation axis. A cylindric panorama is partitioned into image columns of equal width which are parallel to the rotation axis. W is the number of image columns. There is a one-to-one ordered mapping between image columns and projection centers. The distance between a projection center and its associated image column is the effective focal length f . The principal angle ω is between a projection ray in the base plane, emitting from C, and the normal vector of the base circle at point C. R, f , ω, and W are the four intrinsic sensor parameters, characterizing a panoramic image EP(R, f , ω, W ). The affine transform between two sensor coordinate systems is described by a 3× 3 rotation matrix R = [r1 r2 r3 ] and a 3× 1 translation vector T = (tx, ty, tz). Pose Estimation for Sensors Which Capture Cylindric Panoramas 3 Fig. 2. Row difference 4y between the actual corresponding image point (x2, y2) and the point where epipolar curve and column x2 intersect. Let (x1, y1) and (x2, y2) denote the image coordinates of the projection of a 3D point in two panoramas EP1(R1, f1, ω1, W1) and EP2(R2, f2, ω2, W2), respectively. If multiple pairs of corresponding image points are provided, say (x1i, y1i) and (x2i, y2i), for i = 1, 2, . . . , n, then we are able to estimate sensor poses by minimizing the following cost function,