结合多尺度与可变形卷积的自监督图像特征点提取网络

A Self-supervised Feature Points Extraction Networks based on Multi-scale and Deformable Convolution

摘要:

特征点提取是图像处理领域的一个重要方向，在视觉导航、图像匹配、三维重建等领域具有广泛的应用价值。基于卷积神经网络的特征点提取方法是目前的主流方法，但由于传统卷积层的感受野大小不变、采样区域的几何结构固定，在尺度、视角和光照变化较大的情况下，特征点提取的精度和鲁棒性较差。为解决以上问题提出了一种结合多尺度与可变形卷积的自监督特征点提取网络。本文以L2-NET为网络骨干，在深层网络中引入多尺度卷积核，增强网络的多尺度特征提取能力，获得细粒度尺度信息的特征图；使用单应矩阵约束的可变形卷积以提取不规则的特征区域，同时降低运算量，并采用归一化约束单应矩阵的求解，均衡不同采样点对结果的影响，配合在网络中增加的卷积注意力机制和坐标注意力机制，提升网络的特征提取能力。文章在HPatches数据集上进行了对比试验和消融实验，与R2D2等7种主流方法进行对比，本文方法的特征点提取效果最好，相比于次优数据，特征点重复度指标（Rep）提升了约1%，匹配分数（M.s.）提升了约1.3%，平均匹配精度（MMA）提高了约0.4%。本文提出的方法充分利用了可变形卷积提供的深层信息，融合了不同尺度的特征，使特征点提取结果更加准确和鲁棒。

Abstract:

Feature point extraction is an important direction in the field of image processing. It has a wide range of applications in the fields of visual navigation, image matching, 3D reconstruction and so on. The feature point extraction method based on convolution neural network is the mainstream method at present. However, due to the constant size of the receptive field of the traditional convolution layer and the fixed geometric structure of the sampling area, the accuracy and robustness of feature point extraction are poor when the scale, viewing angle and illumination change greatly. In this paper, a self supervised feature point extraction network combining multi-scale and deformable convolution is proposed. Taking l2-net as the backbone of the network, this paper introduces multi-scale convolution kernel into the deep network to enhance the multi-scale feature extraction ability of the network and obtain the feature map of fine-grained scale information; Deformable convolution constrained by homography matrix is used to extract irregular feature regions, reduce the amount of computation, and solve the normalized constrained homography matrix to balance the impact of different sampling points on the results, cooperate with the convolution attention mechanism and coordinate attention mechanism added in the network to improve the feature extraction ability of the network. In this paper, comparative experiments and ablation experiments are carried out on hpatches data set. Compared with seven mainstream methods such as R2D2, the feature point extraction effect of this method is the best. Compared with suboptimal data, the feature point repeatability index (REP) is improved by about 1%, the matching score (M.S.) is improved by about 1.3%, and the average matching accuracy (MMA) is improved by about 0.4%. The method proposed in this paper makes full use of the deep information provided by deformable convolution and integrates the features of different scales to make the feature point extraction results more accurate and robust.