面向物流机器人的改进Q-Learning动态避障算法研究

Improved Q-Learning Dynamic Obstacle Avoidance Algorithm for Logistics Robots

摘要:

为提升物流机器人(Autonomous Mobile Robot,AMR)在复杂环境中的自主导航与避障能力,改善传统Q-Learning算法在动态环境中的收敛速度慢、路径规划不够优化等问题,对面向AMR的Q-Learning算法进行搜索方向和路径节点改进,并引入动态窗口法(Dynamic Window Approach,DWA)算法对进行路径节点和平滑加速改进,实现局部路径规划,以提高改进Q-Learning算法在AMR动态避障中的搜索性能和效率。结果表明,改进Q-learning算法能有效优化搜索路径,能较好避开动态障碍物和静态障碍物,与其他算法的距离差幅至少大于1m；改进算法在局部路径中的避障轨迹更趋近于期望值,最大搜索时间不超过3s,优于其他算法,且其在不同场景下的避障路径长度和运动时间减少幅度均超过10%,避障成功率超过90%。研究方法能满足智慧仓储、智能制造等工程领域对物流机器人高效、安全作业的需求。

Abstract:

In order to improve the Autonomous navigation and obstacle avoidance ability of Autonomous Mobile Robot (AMR) in complex environment, and improve the slow convergence speed and insufficient optimization of path planning of traditional Q-Learning algorithm in dynamic environment, Improve the search direction and path node of the AMR oriented Q-Learning algorithm, and introduce the Dynamic Window Approach (DWA) algorithm to improve path node and smooth acceleration, and realize local path planning. To improve the search performance and efficiency of improved Q-Learning algorithm in AMR dynamic obstacle avoidance. The results show that the improved Q-learning algorithm can effectively optimize the search path, avoid dynamic and static obstacles better, and the distance difference between the improved Q-learning algorithm and other algorithms is at least 1m. The obstacle avoidance trajectory of the improved algorithm in the local path is closer to the expected value, and the maximum search time is no more than 3s, which is better than other algorithms. In addition, its obstacle avoidance path length and movement time are reduced by more than 10% in different scenarios, and the obstacle avoidance success rate is more than 90%. The research method can meet the demand for efficient and safe operation of logistics robots in intelligent warehousing, intelligent manufacturing and other engineering fields.