基于Q学习的水下滑翔机路径规划方法

Path planning method of underwater glider based on Q-Learning

摘要:

针对水下滑翔机路径规划问题，提出了一种基于Q学习的水下滑翔机路径规划方法。考虑到水下滑翔机在执行一些特定任务时会提前给定俯仰角及深度参数，且航向角选择范围通常是几个离散角度值，本文针对典型的几种俯仰角情况分别设计了航向动作选择集，这避免了Q学习方法“维数爆炸”问题。根据水下滑翔机航程最短的目标和障碍物外部约束条件，设计了奖励函数与动作选择策略。相较于传统路径规划方法，本文提出的方法不需要提前知道环境信息，而是在学习过程中根据环境的反馈选择最优动作，因此该方法在不同的环境条件下有优良的迁移能力。仿真结果表明，该方法能在未知环境中为水下滑翔机规划出规避障碍且航程短的路径。

Abstract:

Aiming at the problem of underwater glider path planning, an underwater glider path planning method based on Q-Learning is proposed. Considering that the Underwater Glider will give the pitch angle and depth parameters in advance when performing some specific tasks, and the course angle selection range is usually several discrete angle values, this paper designs the course action selection sets for several typical pitch angles under the condition of certain depth, which avoids the problem of "dimension explosion" of Q-learning method. According to the shortest path target of underwater glider and the external constraints of obstacles, the reward function and action selection strategy are designed. Compared with the traditional path planning methods, the proposed method does not need to know the environmental information in advance, but selects the optimal action according to the feedback of the environment in the learning process. Therefore, this method has excellent migration ability under different environmental conditions. Simulation results show that this method can plan an obstacle avoidance and short range path for underwater glider in unknown environment.