基于改进DDPG算法的双足机器人协同控制

Cooperative control of bipedal robots based on the improved DDPG algorithm

摘要:

针对现有双足机器人协同控制算法存在的轨迹控制偏差大、任务完成耗时长、易与撞障碍物发生碰撞等不足,设计了一种基于改进DDPG算法的双足机器人协同控制方案。构建双足机器人多维度动力学耦合模型,观测在空间范围内机器人运动过程中的坐标变换结果,鉴于多自由度机器人关节运动的分布式关系,引入四元数法分析双足机器人的关节旋转角度变化和空间的姿态变换,对经典DDPG模型进行优化,包括引入双评论家结构解决估值过高问题,基于优先级采样提升对数据的训练效率,利用动态平衡约束奖励函数机制来提升模型的协同控制精度。实际的测试结果显示,改进DDPG算法的轨迹平滑度较好,完成两个场景任务的耗时分别为17.5s和24.0s,优于其他三种协同控制算法,同时在提出算法下未出现与障碍物的碰撞。由此可以得出结论改进算法可解决估值偏差、训练效率低、控制精度不足问题,协同提升机器人运动稳定性。

Abstract:

Aiming at the shortcomings of the existing cooperative control algorithms for bipedal robots, such as large trajectory control deviation, long task completion time, and easy collision with collision obstacles, a cooperative control scheme for bipedal robots based on the improved DDPG algorithm is designed. A multi-dimensional dynamic coupling model of biped robots was constructed to observe the coordinate transformation results during the robot"s movement in the spatial range. Considering the distributed relationship of the joint movement of multi-degree-of-freedom robots, the quaternion method was introduced to analyze the changes in the joint rotation Angle and the spatial attitude transformation of biped robots, and the biped robots were obtained. The DDPG model consists of the policy network, the value network and their respective target networks. It conducts multiple optimizations on the classic DDPG model, specifically including introducing a dual-critic network to address the issue of overvaluation, enhancing the training efficiency of data based on priority sampling, and using a dynamic balance constraint reward function mechanism to improve the collaborative control accuracy of the model. The actual test results show that the trajectory smoothness of the improved DDPG algorithm is better. The time consumption for completing the two scene tasks is 17.5 seconds and 24.0 seconds respectively, which is superior to the three traditional cooperative control algorithms. Meanwhile, no collision with obstacles occurs under the proposed algorithm. From this, it can be concluded that the proposed dual-football co-control scheme has better applicability and co-control effect.