Believable Visual Feedback in Motor Learning Using Occlusion-based Clipping in Video Mapping

Gait rehabilitation systems provide patients with guidance and feedback that assist them to better perform the rehabilitation tasks. Real-time feedback can guide users to correct their movements. Research has shown that the quality of feedback is crucial to enhance motor learning in physical rehabilitation. Common feedback systems based on virtual reality present interactive feedback in a monitor in front of a user. However, in this technique, there is a gap between where the feedback is presented and where the actual movement occurs. In particular, there is a discrepancy between where the actual movement occurs (e.g., on a treadmill) and the place of presenting feedback (e.g., a screen in front of the user). As a result, the feedback is not provided in the same location, which requires users perform additional cognitive processing to understand and apply the feedback. This discrepancy is misleading and can consequently result in difficulties to adapt the changes in rehabilitation tasks. In addition, the occlusion problem is not well handled in existing feedback systems that results in misleading the users to assume that the obstacle is on the foot. To address this problem, we need to make an illusion of putting a foot on the obstacle. In this paper, we propose a visual feedback system based on video mapping to provide a better understanding of the relationship between body perception and movement kinematics. This system is based on Augmented Reality (AR) in which visual cues in the form of light are projected on the treadmill using video projectors. In this system, occlusion-based clipping is used to enhance the believability of the feedback. We argue that this system contributes to the correct execution of rehabilitation exercises by increasing patients’ awareness of gait speed and step length. We designed and implemented two prototypes including the video projection with occlusion-based clipping (OC) and a prototype with no occlusion-based clipping (NOC). A set of experiments were performed to assess and compare the ability of unimpaired participants to detect real-time feedback and make modifications to gait using our feedback system. In particular, we asked 24 unimpaired participants to perform stepping and obstacle avoidance tasks. Since the focus of the paper is the quality of the feedback than the effect of feedback on training in long-term, unimpaired participants were recruited for this study. In the experiments, a motion capture device was used to measure the performance of participants. We demonstrated that our system is effective in terms of steps to adapt changes, obstacles to adapt changes, normalized accumulative deviation, quality of user experience, and intuitiveness of feedback. The results showed that projection-based AR feedback can successfully guide participants through a rehabilitation exercise. In particular, the results of this study showed statistically significant differences between the fault-rate of participants using OC and NOC prototypes in the stepping (p=0.0031) and obstacle avoidance (0.021) tasks. In addition, participates rated OC more intuitive than NOC in terms of the quality of feedback. Our feedback system showed a significant improvement in participants’ ability to adapt the changes while walking on the treadmill.