基于双阈值的ANN-SNN转换方法优化

Optimization of ANN-SNN Conversion Method With Double Threshold

摘要:

脉冲神经网络作为第三代神经网络，能够克服许多人工神经网络中所存在的问题，如高功耗、鲁棒性较差等。通过对预训练好的人工神经网络模型进行转换是获取深度脉冲神经网络模型的一种主要方法，然而通过这种方法获取的脉冲神经网络的延迟较高，无法满足实时性要求。论文在双阈值转换方法的基础上，采用阈值平衡技术对转换过程进行优化，通过理论推导，提出了一种对称阈值LeakyReLU激活函数，并对人工神经网络到脉冲神经网络的转换流程进行了梳理。此外，采用了泄漏机制对转换后的脉冲神经网络模型结构进行了优化，并通过脉冲时序依赖可塑性学习规则对该结构进行训练。最终，在MNIST数据集与CIFAR-10数据集上进行了实验，结果表明，优化后脉冲神经网络的收敛速度与鲁棒性得到了大幅提升。

Abstract:

As a third-generation neural network, spiking neural network can overcome many problems in artificial neural networks, such as high power consumption and poor robustness. Transforming the pre-trained artificial neural network model is one of the main methods to obtain the deep spiking neural network model, but the spiking neural network obtained by this method has a high latency and cannot meet the real-time requirements. On the basis of the double threshold conversion method, the threshold balance technology is used to optimize the conversion process, and through theoretical derivation, a symmetric threshold LeakyReLU activation function is proposed, and the conversion process from artificial neural network to spiking neural network is sorted out. In addition, the leakage mechanism is used to optimize the structure of the transformed spiking neural network model, and the structure is trained by the spike-timing-dependent plasticity learning rule. Finally, experiments were carried out on the MNIST dataset and the CIFAR-10 dataset, and the results showed that the convergence speed and robustness of the optimized spiking neural network were greatly improved.