基于Q学习的供热末端自适应PID控制算法

Heating end adaptive PID control algorithm based on Q learning

摘要:

城市建筑集中供热末端“全开”和“全关”控制方式不仅热舒适性差,也造成了较大热损耗。为改善这一问题,在研究了强化学习的基础上提出基于Q学习PID参数的供热末端流量控制算法。首先分析了散热器等的热动态及传热过程,建立了供热房间热平衡数学模型,然后以PID控制算法为基础,温度偏差为控制器输入,调节阀开度控制量为输出,选择温差变化为智能体奖惩的学习策略,通过Q学习算法对PID参数进行在线自适应整定,最后在集中供热末端流量调节的仿真实验中验证了控制器的调控性能并与传统PID控制结果进行了对比。实验结果表明,基于Q学习的自适应PID流量控制算法能够使室内温度变化和调节阀开度变化更加平缓,且节省约33%的供热量,节能效果较明显。

Abstract:

In the central heating of urban buildings, the “full open” and “fully closed” control modes at the end of heating not only have poor thermal comfort, but also cause large heat loss. For improving this problem, This paper proposes that the heating end flow control algorithm based on Q learning PID parameters is based on the study of reinforcement learning. First, we analyzed the thermal dynamics and heat transfer process of the radiator, and established a mathematical model of heat balance in the heating room. Then, the temperature deviation is the controller input, and the control valve opening degree control quantity is output based on the PID control algorithm. The temperature difference change is selected as the learning strategy of the intelligent body reward and punishment, and the PID parameters are adaptively adjusted online by the Q learning algorithm. Finally, we verified the controller"s regulation performance in the simulation experiment of central heating end flow regulation and compared with the traditional PID control results. The experimental results show that the adaptive PID flow control algorithm based on Q learning can make the indoor temperature change and the adjustment valve opening change more gradual, and save about 33% of the heat supply, and the energy saving effect is more obvious.