Immersive Real‐Acting Space with Gesture Tracking Sensors

In this study, we realized a real space based virtual aquarium equipped with a multi‐view function that provides images for users and observers at the same time. A virtual reality system needs more natural and intuitive interfaces so as to enhance users’ immersion. We attach markers on users and camera devices in a real space designed in the same 1:1 size as the virtual space to collect motion information, which is reflected in real time to generate virtual world images, which in turn are transmitted to the users’ immersing image devices. Also, the system allows observers to share experiences by providing them virtual synthetic images from a third person perspective including a user after filming the user in the real space with a camcorder on which motion tracking markers are attached. For this, the system provides the functions of marker‐based motion tracking, recognition of users’ motions, real time actual image synthesis, and multi‐view to realize a method to simulate more intuitive and natural virtual space interactions, which can be used for the construction of motion‐based realistic/experiencing systems, which increasingly attract interest.