Towards an assembly plan from observation : fine localization based on face contact constraints

We have been developing a novel method to program a robot, an APO (assemblyplan-from-observation) method. A human performs assembly operations in front of the APO system's TV camera. The APO system recognizes such assembly operations and generates an assembly plan to repeat the assembly operations using its robot arm. Since an important purpose of assembly operations is to achieve face contacts between objects, our previous system was based on face contact relations. Our system recognized object configurations after each of assembly operation and then extracted face contact relations from the observed object configurations. By associating a face contact relation with the operation necessary to achieve the relation, the system was able to construct a plan to repeat the assembly. In this system, two kinds of information were extracted from object configurations: 1) face-contact relation and 2) motion parameters necessary to move objects around. The system works well while noise-free input. Usually, however, object configurations contain some degree of error. Since a face contact relation is a topological relation, it can be obtained reasonably well from noisy object recognition results. However, motion parameters are obtained directly by converting object configurations. Under the presence of error, due to error-contaminated motion parameters, the system may fail to perform an assembly operation, although a face-contact relation and thus an assembly operation is correctly recovered. This paper proposes a method to correct erroneous motion parameters based on a face contact relation. We assume that a given face contact relation reflects the actual face contact correctly. Face contact constraint equations will be defined for each pair of contact faces. A face contact equation requires that a vertex of one face is on the plane including the other face. Motion parameters are determined by solving face contact constraint equations simultaneously using the singular value decomposition method. In order to maintain the relationship among motion parameters of previous operations, we define operation dependency lists (ODL), symbolic lists of homogeneous transformations to represent operations. An ODL is calculated for each assembly operation, and is attached to each object. By using ODLs instead of object configurations, we can apply the same method to obtain correct motion parameters for former operations. We implement this method in the APO system, apply the method to several assembly examples, and verify the effectiveness of the method. University Libraries Carnegie We!lon University Pittsburgh Pennsylvania