基于双流卷积神经网络的改进人体行为识别算法

An Improved Algorithm of Human Action Recognition Based on Two-Stream Convolutional Networks

摘要:

近年来人体行为识别成为计算机视觉领域的一个研究热点,而卷积神经网络(Convolutional Neural Network,CNN)在图像分类和识别领域取得了重要突破,但是人体行为识别是基于视频分析的,视频包含空间域和时间域两部分的信息。针对基于视频的人体行为识别问题,提出一种改进的双流卷积神经网络(Two-Stream CNN)模型,对于空间域,将视频的单帧RGB图像作为输入,送入VGGNet_16模型；对于时间域,将多帧叠加后的光流图像作为输入,送入Flow_Net模型；最终将两个模型的Softmax输出加权融合作为输出结果,得到一个多模型融合的人体行为识别器。基于JHMDB公开数据库的实验,结果证明了改进的双流CNN在人体行为识别任务上的有效性。

Abstract:

In recent years, human action recognition has become a research hotspot in the field of computer vision. The research on Convolutional Neural Network has made great breakthroughs in the field of image classification and recognition, but human action recognition is based on video, containing information in both spatial domain and temporal domain. Aiming at human action recognition based on video, an improved Two-Stream ConvNet architecture is proposed. For the spatial domain, the single RGB image is fed into the VGGNet_16 model. For the temporal domain, the superposition of multi optical flow images is fed into the Flow_Net model. Finally, the softmax outputs of the two models are merged with the linear weighting to realize human action recognition. The experiments based on the JHMDB public database prove the effectiveness of the improved Two-Stream ConvNet architecture on human action recognition.