Multi-Fingered In-Hand Manipulation With Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors

Multi-fingered hands could be used to achieve many dexterous manipulation tasks, similarly humans, and tactile sensing enhance the stability for a variety of objects. However, sensors on multi-fingered have sizes shapes. Convolutional neural networks (CNN) can useful processing information, but information from needs an arbitrary pre-processing, as CNNs require rectangularly shaped input, which may lead unstable results. Therefore, how process such complex utilize it achieving skills is still open issue. This paper presents control method based graph convolutional network (GCN) extracts geodesical features data with complicated sensor alignments. Moreover, object property labels are provided GCN adjust in-hand motions. Distributed tri-axial mounted fingertips, finger phalanges palm Allegro hand, resulting in 1152 measurements. Training collected data-glove transfer human directly robot hand. The achieved high success rates manipulation. We also confirmed that fragile objects were deformed less when correct GCN. When visualizing activation PCA, we verified acquired features. Our stable even experimenter pulled grasped untrained