From Model to Appearance Based Autism Spectrum Disorder Classification: The Joint Energy Image

In appearance-based gait analysis studies, Gait Energy Images (GEI) have been shown to be an effective tool for human identification and gait pathology detection. Alternatively, model-based studies found kinematic and spatiotemporal features to be useful for gait recognition and Autism Spectrum Disorder (ASD) classification. This paper combines elements of these techniques within the ASD classification problem by averaging binary images representing a person’s joint positions throughout their gait cycle. This is compared to the use of full body silhouettes that are used in GEIs. Depth is encoded into the binary images before they are averaged using colour mapping, a technique used in the Chrono-Gait Image. The Joint Energy Image (JEI) therefore embeds both temporal and depth information of the joints into a 2D image. The image was preprocessed using Principal Component Analysis before being applied to a Multi-
Layer Perceptron, and a Random Forest classifier. The JEI was also applied to a Convolutional Neural Network directly. Accuracy was improved by using a Test Time Augmentation measure where the original and augmented JEIs were used in a voting system to obtain a prediction. The CNN achieved the optimal Test Time Augmented (TTA) accuracy of 95.56% when trained on the primary dataset of 100 subjects (50 with ASD and 50 that are typically developed), and 80% TTA accuracy on the secondary dataset of 20 subjects (10 ASD and 10 typically developed) across multiple tests, thus showing the viability of both the CNN and the JEI to compete with state-of-the-art feature sets and performances.