基于Dueling Double DQN的交通信号控制方法

Traffic Signal Control Method based on Dueling Double DQN

摘要:

为了提高交叉口通行效率缓解交通拥堵，深入挖掘交通状态信息中所包含的深层次隐含特征信息，提出了一种基于Dueling Double DQN (D3QN) 的单交叉口交通信号控制方法。构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型，对动作-价值函数的估计值和目标值迭代运算过程进行了优化，克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题。设计了一个新的Dueling Network解耦交通状态和相位动作的价值，增强Double DQN (DDQN) 提取深层次特征信息的能力。基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境，开展仿真测试。仿真测试结果表明，与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比，所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数，明显提升交叉口通行效率。

Abstract:

In order to improve the efficiency of intersection traffic and alleviate traffic congestion, a single intersection traffic signal control method based on Dueling Double DQN (D3QN) is proposed by deeply exploring the deep hidden feature information contained in traffic status information. A traffic signal control model based on deep reinforcement learning Double DQN (DDQN) was constructed, and the iterative operation process of the estimated value and target value of the action value function was optimized to overcome the problem of slow convergence speed in traffic signal control models based on deep reinforcement learning DQN. A new Dueling Network has been designed to decouple the value of traffic states and phase actions, enhancing the ability of Double DQN (DDQN) to extract deep level feature information. A single intersection simulation framework and environment were built based on the micro simulation platform simulation of urban mobility（SUMO）for simulation testing. The simulation test results show that compared with traditional traffic signal control methods and traffic signal control methods based on deep reinforcement learning DQN, the proposed method can effectively reduce the average waiting time, average queue length, and average number of stops of vehicles, significantly improving the efficiency of intersection traffic.