Sign Language Dynamic Gestures Recognition using Depth Data

Abstract

In this report we have proposed a framework for sign language dynamic gestures recognition from depth sequences. For feature representation two different set of features are extracted. First one is gradient local auto correlation features from the depth motion maps and to incorporate the loss of temporal information which is there in depth motion maps, the other set of features extracted is HON4D (Histogram of oriented 4D normal). A new framework for fusing the features at decision level using classifier ensemble of three 2-layer feed forward neural networks is been proposed . The proposed framework is tested on two datasets MSRGesture3D and ISL3D dataset. The ISL3D dataset is created by us having 12 dynamic Indian Sign Language gestures. The recognition accuracies achieved on the two datasets are: 96.99% on MSRGesture3D dataset and 81.38% on ISL3D dataset.