基于深度强化学习的低轨卫星网络算力路由研究

Deep Reinforcement Learning-based Computing Power Routing for Low-Orbit Satellite Networks

摘要:

面向未来低轨卫星计算、网络等资源联合调度与优化需求，提出一种基于深度强化学习的低轨算力路由方案，能够解决低轨卫星网络多维资源协同效率低、利用率低下的问题。基于算网编排控制器的算力路由协议流程，建立了时延最优的算力调度优化模型，设计并实现了一种基于DQN的低轨算力路由智能算法，将低轨卫星算力路由寻址建模为马尔可夫决策过程，定义了包含业务、拓扑、算力等特征的状态空间和与时延最优相关的奖励函数。经过模型训练和仿真分析，收敛后的智能算法与基准算法相比，能够显著提高计算资源和网络资源的综合利用效率，降低任务处理所需时间，优化用户体验。

Abstract:

Aiming at the future demand for joint scheduling and optimization of computing and networking re-sources in low Earth orbit (LEO) satellite systems, a deep reinforcement learning-based LEO computing power routing scheme is proposed to address the low efficiency and utilization of multi-dimensional re-source collaboration in LEO satellite networks. Based on the computing power routing protocol of a computing and networking orchestration controller, an optimal delay model for computing power sched-uling is established. Additionally, an intelligent algorithm for LEO computing power routing based on Deep Q-Network is developed and implemented. This algorithm models the LEO satellite computing power routing addressing as a Markov decision process, defining a state space that includes features such as business, topology, and computing power, as well as a reward function related to optimal delay. After model training and simulation analysis, the converged intelligent algorithm significantly improves the comprehensive utilization efficiency of computing and networking resources compared to benchmark al-gorithms, reduces the time required for task processing, and optimizes user experience.