Visual Servoing for a User's Mouth with Effective Intention Reading in a Wheelchair-based Robotic Arm

There exists the co-operative activity between a human being and rehabilitation robots because the human operates rehabilitation robots in the same environment and has the bene t of rehabilitation robots, such as manipulatory or mobile functions. Intention reading is one of the essential functions of humanfriendly rehabilitation robots in order to promise the comfort and safety for all who need them. First of all, the overall structure of a new wheelchair-based robotic arm system, KARES II, and its human-robot interaction technologies are presented. Among those technologies, we concentrate on visual servoing that allows this robotic arm to operate autonomously via visual feedback. E ective intention reading, such as recognizing the positive and negative meaning of the user, is performed on the basis of changes of the facial expression around lips that is strongly related to the user's intention while this robotic arm provides the user with a beverage. For the eÆcient visual information processing, log-polar mapped images are used to control the stereo camera head that is located in the end-e ector of the robotic arm. The visual servoing with e ective intention reading is successfully applied to serve a beverage for the user.