基于强化学习的无人机网络资源分配研究

Research on Resource Allocation in UAV Networks Based on Reinforcement Learning

摘要:

以无人机网络的资源分配为研究对象，研究了基于强化学习的多无人机网络动态时隙分配方案，在无人机网络中，合理地分配时隙资源对改善无人机资源利用率具有重要意义；针对动态时隙分配问题，根据调度问题的限制条件，建立了多无人机网络时隙分配模型，提出了一种基于近端策略优化(PPO)强化学习算法的时隙分配方案，并进行强化学习算法的环境映射，建立马尔可夫决策过程(MDP)模型与强化学习算法接口相匹配；在gym仿真环境下进行模型训练，对提出的时隙分配方案进行验证，仿真结果验证了基于近端策略优化强化学习算法的时隙分配方案在多无人机网络环境下可以高效进行时隙分配，提高网络信道利用率，提出的方案可以根据实际需求适当缩短训练时间得到较优分配结果。

Abstract:

Taking the resource allocation of UAV networks as the research object, a dynamic time slot allocation scheme in multi-UAV networks based on reinforcement learning is investigated. In UAV networks, it is important to reasonably allocate time slot resources to improve UAV resource utilization. Aiming at the dynamic time slot allocation problem, the time slot allocation model of multi-UAV network is established according to the constraints of the scheduling problem. A time slot allocation scheme based on the proximal policy optimization (PPO) reinforcement learning algorithm is proposed. The environment mapping of the reinforcement learning algorithm is also carried out. Build a Markov decision process (MDP) model to match the reinforcement learning algorithm interface. Model training is performed in the gym simulation environment to validate the proposed time slot allocation scheme. The simulation results verify that the time slot allocation scheme based on the proximal policy optimization reinforcement learning algorithm can efficiently perform time slot allocation and improve the network channel utilization in a multi-UAV network environment. The proposed scheme can reduce the training time appropriately to obtain better allocation results according to the actual demand.