基于多智能体强化学习的微装配任务规划方法

Microassembly Task Plaanning Method based on Multi-agent Reinforcement Learning

摘要:

现有装配任务规划方式多为人工规划，存在低效、高成本、易误操作等问题，为此分析了微装配操作的任务特点，以及对微装配中多操作臂协作与竞争关系进行了详细分析，并提出多智能体强化学习中符合微装配任务特点的动作空间、状态空间以及奖励函数的构建方法；利用CoppeliaSim仿真软件构建合理的仿真模型，对已有设备进行物理建模，构建了基于多智能体深度确定性策略梯度算法的学习模型并进行训练，在仿真环境中对设计的状态、动作空间以及奖励函数进行了逐项实验验证，最终获得了稳定的路径以及完整的任务实施方案；仿真结果表明，提出的环境构建方法，更契合直角坐标运动为主要框架的微装配任务，能够克服现有规划方法的不足，能够实现可实际工程化的多臂协同操作，提高任务的效率以及规划的自动化程度。

Abstract:

The existing planning methods mostly are manual planning, which have problems such as inefficiency, high cost, and easy misoperation. Thus, the characteristics of microassembly operation tasks, collaboration and competition relationship of micro-assembly operation are analyzed in detail, and a method for the construction of action, state and reward conditions that conforms to the characteristics of micro-assembly tasks in multi-agent reinforcement learning is proposed. Using CoppeliaSim simulation software to model existing equipment physically, a learning model is built and trained based on multi-agent deep deterministic policy gradient algorithm, then the designed action, state and reward function are verified experimentally in simulation environment. Ultimately a stable path and complete task implementation scheme is obtained. The simulation results show that the proposed method is more suitable for the micro-assembly task with Cartesian coordinate motion as the main framework, and can overcome the shortcomings of existing planning methods. Besides, the method can realize the multi manipulator arm collaborative operation, which can be practically engineered and improve the efficiency of the task and the automation degree of planning.