Abstract

Sign language is the natural way of communication of speech and hearing-impaired people. Using Indian Sign Language (ISL) interpretation system, hearing impaired people may interact with normal people with the help of Human Computer Interaction (HCI). This paper presents a method for automatic recognition of two-handed signs of Indian Sign language (ISL). The three phases of this work include preprocessing, feature extraction and classification. We trained a BPN with Histogram Oriented Gradient (HOG) features. The trained model is used for testing the real time gestures. The overall accuracy achieved was 89.5% with 5184 input features and 50 hidden neurons. A deep learning approach was also implemented using AlexNet, GoogleNet, VGG-16 and VGG-19 which gave accuracies of 99.11%, 95.84%, 98.42% and 99.11% respectively. MATLAB is used as the simulation platform. The proposed technology is used as a teaching assistant for specially abled persons and has demonstrated an increase in cognitive ability of 60–70% in children. This system demonstrates image processing and machine learning approaches to recognize alphabets from the Indian sign language, which can be used as an ICT (information and communication technology) tool to enhance their cognitive capability.