127 / 2022-10-21 10:17:32

Dynamic Gesture Recognition Based on Temporal Shift Module

Dynamic gesture recognition is a very important interaction method in human-computer interaction. For the current research, multi-modal data and three-dimensional convolutional neural network are often used for training. Although the recognition accuracy is high and robustness is good, the amount of parameters is large and high computational cost.To solve the problem, a dynamic gesture recognition method based on Temporal Shift Module (TSM) is proposed on the basis of two-dimensional convolutional neural network. This method uses PyConvResNet-50 as the backbone network, adds the TSM module for information exchange in the time dimension, embeds the motion excitation module (ME) into the TSM to enhance short-term temporal modeling, and finally uses 2D-FCN for spatiotemporal feature fusion classification. The experimental results show that the recognition accuracy of the model on the large-scale gesture dataset Jester is 96.49%, which is comparable to that of the three-dimensional convolutional neural network, but the calculation amount is reduced by 63% as well. This method is suitable for the field of gesture recognition that requires high real-time performance.