DTF-VTNet:基于动态特征融合的机器人视触觉滑移检测方法

<p>DTF-VTNet: A Dynamic Feature Fusion-Based Method for Robot Visual-Tactile Slip Detection

摘要:

在机器人抓取领域中，准确评估手持物体是否发生滑移是实现稳定操纵的重要基础；视觉图像能够反映物体姿态及整体运动变化，触觉信息能够表征夹爪与物体接触过程中的局部形变和压力变化，二者融合有助于提升滑移检测性能；然而，滑移现象通常只在关键时刻出现，且视觉与触觉信息在表达方式上存在差异，简单拼接或固定融合方式难以充分突出与滑移相关的关键时刻特征因此导致滑移检测准确率低；针对上述问题，提出一种动态特征融合视触觉网络（DTF-VTNet，dynamic feature fusion visual-tactile network）；该方法首先通过通道混洗和双向交叉注意力机制实现视觉与触觉模态对齐；随后设计动态特征融合模块，联合建模模态内时序重要性与跨模态一致性，自适应筛选有效特征并抑制低贡献时序片段干扰；最后利用多尺度时序卷积网络建模抓取过程中的连续状态变化，并通过时序注意力池化聚合关键时刻特征，实现滑移状态分类；实验结果表明，所提方法在公开视觉-触觉滑移检测数据集上取得 91.45% 的准确率，验证了其在视触觉融合与滑移检测任务中的有效性。

Abstract:

Accurate slip detection is essential for stable robotic grasping. Visual images reflect object pose and motion changes, while tactile information captures local deformation and pressure variations during contact. Although visual-tactile fusion can improve detection performance, slip events usually occur only at critical moments, and the representation gap between visual and tactile modalities makes simple concatenation or fixed fusion strategies insufficient for highlighting slip-related features. To address this problem, this paper proposes a dynamic feature fusion visual-tactile network (DTF-VTNet) for robotic slip detection. The method first employs channel shuffle and bidirectional cross-attention to align visual and tactile modalities. Then, a dynamic feature fusion module jointly models intra-modal temporal importance and cross-modal consistency to adaptively select discriminative features and suppress low-contribution temporal segments. Finally, a multi-scale temporal convolutional network and temporal attention pooling are used to model continuous grasping state changes and aggregate key-moment features for slip classification. Experimental results on a public visual-tactile slip detection dataset show that DTF-VTNet achieves an accuracy of 91.45%, demonstrating its effectiveness in visual-tactile fusion and slip detection.