A Novel Pipeline for Object Recognition Utilising Multi-Sensory Tactile Fusion

Conventional visual-based object recognition is subject to many variables which may cause degradation, such as improper illumination and occlusion. Tactile sensing-based object recognition can assist in situations where these issues occur, enabling a system to exploit features that standard visual systems cannot identify. Tactile sensing-based object recognition involves the gathering and processing of physical features related to the interaction between a tactile sensing system such as a robot, and a physical object. This work proposes a novel object recognition pipeline driven by a multi-sensory tactile fusion model based on the state-of-the-art time-series classifier, MiniROCKET. It builds upon the authors' previously published research, which achieved state-of-the-art performance for single-modality tactile object recognition, and by implementing a collection of classification heads on both the ROCKET and MiniROCKET pipelines. This work demonstrates how the combination of multiple tactile sensing modalities can achieve excellent performance, exceeding the performance of current systems which use a combination of both visual and tactile systems. This research achieves a state-of-the-art performance on the PHAC-2 dataset, exceeding what was previously achieved in accuracy by 3.3% while simultaneously reducing computational costs by up to 90%.