基于增强现实与手势识别算法的煤矿掘进设备远程监测模型研究

Design of remote monitoring model for coal mine excavation equipment based on augmented reality and gesture recognition algorithm

摘要:

为提升煤矿掘进设备远程监测的实时性、交互性与环境适应性,解决传统检测手段在立体感知及动态响应方面的不足,采用SICK Visionary-T Mini AP摄像机采集RGB与深度数据,经点云渲染、三维重构实现高精度虚实融合；结合r700 AR设备与基于信道状态信息(CSI)的多分支卷积门控循环神经网络(Multi-branch CNN-GRU),实现远程手势交互控制。在粉尘浓度50~150mg/m3、光照强度 50~500lux 及静动态工况下实验验证,结果显示：静态理想工况下AR重构RMSE为1.67~2.71mm,复杂动态工况下低于6.63mm；动态环境中手势识别率达99.6%以上,操作误差控制在0.05~0.5cm,端到端操作延迟85~120ms。该模型鲁棒性强、工程实用性佳,为煤矿掘进设备远程实时监测与交互控制提供可靠技术支撑。

Abstract:

[Objective]To enhance the real-time performance, interactivity, and environmental adaptability of remote monitoring for coal mine excavation equipment, and to overcome the limitations of traditional detection methods in terms of three-dimensional perception and dynamic response. [Method]The SICK Visionary-T Mini AP camera was employed to collect RGB and depth data, and high?precision virtual?real fusion was achieved through point cloud rendering and three?dimensional reconstruction. Combined with the r700 AR device and a multi?branch convolutional gated recurrent neural network (Multi?branch CNN?GRU) based on channel state information (CSI), remote gesture interaction control was realized. [Results]Experiments were conducted under dust concentrations of 50–150?mg/m3, light intensities of 50–500?lux, and both static and dynamic working conditions. The results indicate that the AR reconstruction RMSE ranges from 1.67?mm to 2.71?mm under ideal static conditions, and remains below 6.63?mm under complex dynamic conditions. In dynamic environments, the gesture recognition rate exceeds 99.6%, the operation error is controlled within 0.05–0.5?cm, and the end?to?end operation delay is 85–120?ms.[Conclusion]The proposed model exhibits strong robustness and good engineering practicality, providing reliable technical support for remote real?time monitoring and interactive control of coal mine excavation equipment.