EgoFish3D: Egocentric 3D Pose Estimation from a Fisheye Camera via Self-Supervised Learning

Egocentric vision has gained increasing popularity recently, opening new avenues for human-centric applications. However, the use of egocentric fisheye cameras allows wide angle coverage but image distortion is introduced along with strong human body self-occlusion imposing significant challenges in data processing and model reconstruction. Unlike previous work only leveraging synthetic training, this paper presents a real-world EgoCentric Human Pose (ECHP) dataset. To tackle difficulty collecting 3D ground truth using motion capture systems, we simultaneously collect images from head-mounted camera as well two third-person-view cameras, circumventing environmental restrictions. By self-supervised learning under multi-view constraints, propose simple yet effective framework, namely EgoFish3D, pose estimation single different scenarios. The proposed EgoFish3D incorporates three main modules. 1) The module takes exocentric input estimates represented third-person frame; 2) xmlns:xlink="http://www.w3.org/1999/xlink">the predicts 3) interactive rotation matrix between views. Experimental results on our ECHP dataset existing benchmark datasets demonstrate effectiveness which can achieve superior performance to methods.