A Vision-Based Approach for Indian Sign Language Recognition

The sign language is the essential communication method between the deaf and dumb people. In this paper, the authors present a vision based approach which efficiently recognize the signs of Indian Sign Language (ISL) and translate the accurate meaning of those recognized signs. A new feature vector is computed by fusing Hu invariant moment and structural shape descriptor to recognize sign. A multi-class Support Vector Machine (MSVM) is utilized for training and classifying signs of ISL. The performance of the algorithm is illustrated by simulations carried out on a dataset having 720 images. Experimental results demonstrate that the proposed approach can successfully recognize hand gesture with 96% recognition rate. DOI: 10.4018/ijcvip.2012100103 26 International Journal of Computer Vision and Image Processing, 2(4), 25-36, October-December 2012 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. There are two main approaches used in the sign language recognition that is Glove/Device based and Vision based (Ong & Ranganath, 2005). In the glove based method the user has to wear a device which carries a load of cables so as to connect the device to a computer. Such devices are expensive and reduce the naturalness of the sign language communication. In contrast, the Vision based method requires only a camera and directly deals with image gestures. It is a two step process: sign capturing and sign analysis. Vision based methods provide a natural environment to the user and reduces the complications as in the glove based method. Extraction of visual information in the form of feature vector is an important part in gesture recognition problem (Ong & Ranganath, 2005). However, there are challenges like tracking of hand, segmentation of hand from the background and environment, illumination variation, occlusion, movements, and position (Stefan, Wang, & Athitsos, 2009). Every country has its own sign language with a high level of grammatical variations. The sign language exists in India is commonly known as Indian Sign Language (ISL). It has been argued that perhaps the same sign language is used in Nepal, Sri Lanka, Bangladesh, and border regions of Pakistan (Dasgupta, Shukla, Kumar, Diwakar, & Basu, 2008). Examples of other sign languages are the American Sign Language (ASL), the British Sign Language (BSL), the Korean Sign Language (KSL), and so on. The All India Federation of the Deaf estimates around 4 million deaf people and more than 10 million hard of hearing people in India (Dasgupta et al., 2008). Studies revealed that, one out of every five deaf people in the world are from India. More than 1 million deaf adults and around 0.5 million deaf children uses Indian Sign Language as a mode of communication and most of the systems developed for sign language recognition are native language specific and hence, cannot be used for ISL (Dasgupta et al., 2008). In general, the semantic meaning of the language components in all sign languages differs, however there are signs with a universal syntax. For instance, a simple gesture by one hand expressing ‘hi’ or ‘goodbye’ has the similar meaning all over the world and in all forms of sign languages. ISL is a complete natural language, found in India with its own morphology, phonology, syntax, and grammar (Dasgupta et al., 2008). Indian Sign Language (ISL) is a visual-spatial language which provides linguistic information by using hand, arms, face, and head/body gestures. ISL produces both isolated as well as continuous signs. An isolated sign focuses on a single hand gesture, and is an exacting hand configuration and pose represented by a particular image. A continuous sign is a moving gesture, represented by series of images. In earlier work on ISL recognition, Agrawal et al., (2011) proposed the use of a ‘two stage classification’. In stage-I, gestures with high degree of errors was removed and stage-II was used to recognize computationally intensive features. This approach can be used to give an evaluation to the user about the correctness of the sign performed and also uses the latex gloves which increased the cost-effectiveness of the system with a challenge of color segmentation. Futane and Dharaskar (2011) developed a simple ISL recognition system by exploiting hand geometry features. However, this approach contains the dialects of ISL from American or British sign language. Panwar and Mehra (2011) proposed the hand gesture recognition approach based on detection of shape based features. However, this approach does not follow any systematic method to define certain parameters for gesture recognition and all the threshold values are based on assumption. Bhuyan et al. (2006) proposed a dynamic hand gesture recognition system through Finite State Machine. However, the gesture recognizer can recognize only some hand gestures from ISL where other body parts bears no information for selected gestures. The objective of the proposed work is to efficiently recognize the signs of Indian Sign Language (ISL) and translate the accurate meaning of those recognized signs. In this paper, we have proposed an Indian Sign Lan10 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the product's webpage: www.igi-global.com/article/vision-based-approach-indiansign/75768?camid=4v1 This title is available in InfoSci-Journals, InfoSci-Journal Disciplines Computer Science, Security, and Information Technology. Recommend this product to your librarian: www.igi-global.com/e-resources/libraryrecommendation/?id=2