基于双模态门控特征融合的跌倒检测方法

Fall Detection Method Based on Bi-modal Gated Feature Fusion

摘要:

使用单一传感器进行人体跌倒检测的方法不能充分捕捉动作特征,摄像头在光线较差时无法获得高质量图像,毫米波雷达的点云稀疏性降低了远距离目标信息的有效性；针对上述问题,提出了一种基于双模态门控特征融合的跌倒检测方法；使用雷达和摄像头同步检测,雷达分支根据时间-距离图和微多普勒图获得融合特征,视觉分支提取目标的光学特征；将两种特征送入门控融合模块,根据权重整合特征信息,在输出层实现分类；设计了雷达分支和整体网络的相关实验,雷达分支融合方法的平均准确率是91.7%,优于单一特征方法；整体网络的门控融合方法的准确率是94.1%,相比特征相加融合和首尾拼接融合方法分别高出3.0%和1.8%；充分表明该方法能够提升人体跌倒检测的性能。

Abstract:

The method of human fall detection using a single type of sensor cannot adequately capture motion features. The camera cannot obtain high-quality images under poor lighting conditions, and the point cloud sparsity of millimeter-wave radar reduces the effectiveness of remote target information. To solve these problems, a fall detection method based on bi-modal gated feature fusion is proposed. The method uses radar and camera to synchronized detection. The radar branch obtains the fusion feature based on time-distance map and micro-Doppler map, and the visual branch extracts the optical feature of the target. The two features are sent to the gated fusion module, and the feature information is integrated according to the weight to realize fall detection at the output layer. Experiments of the radar branch and the overall network are designed. The average accuracy of the radar branch fusion method is 91.7%, which is better than that of the single feature method. The accuracy of gated fusion method in the overall network is 94.1%, which is 3.0% and 1.8% higher than that of feature addition fusion and fore-tail fusion, respectively. It is fully demonstrated that the method can effectively improve the performance of human fall detection.