结合类脑导航的强化学习无人机自主导航

Reinforcement Learning Algorithms Combined with Brain-Inspired Navigation

摘要:

随着无人移动平台的不断发展,为其赋予高效的自主导航能力变得尤为重要；针对无人机自主导航常用的端到端强化学习方法存在训练效率低、泛化能力和通用性差等问题,引入了类脑导航模型,基于长短时记忆(LSTM)神经网络构建了类脑细胞导航模型,通过整合编码无人机智能体的自运动信息,实现了网格细胞和头朝向细胞的编码,进一步将这些信息作为深度强化学习算法D3QN的状态补充表示；通过在AirSim仿真环境的实验表明,类脑导航模型的引入能够有效提高算法的训练能力和无人机智能体的导航性能,相较于原D3QN算法,在环境目标改变的情况下仍能寻找到新的目标点,有效提升了算法的泛化能力。

Abstract:

With the continuous development of unmanned mobile platforms, it has become particularly important to endow them with efficient autonomous navigation capabilities. In response to the low training efficiency, poor generalization ability, and universality of the commonly used end-to-end reinforcement learning methods for autonomous navigation of UAV, a brain-inspired navigation model is introduced. Based on the long short-term memory (LSTM) neural network, a brain-inspired cell navigation model is constructed, which integrates the self-motion information of the UAV intelligent agent to encode grid cells and head direction cells, and further supplements this information as the state of the deep reinforcement learning algorithm D3QN. The experiments in AirSim simulation environment show that the introduction of the brain-inspired navigation model can effectively improve the training ability of the algorithm and the navigation performance of the UAV intelligent agent. Compared with the original D3QN algorithm, it can still find new target points when the environmental target changes, effectively improving the generalization ability of the algorithm.