基于深度强化学习的综合能源业务通道优化机制

doi:10.13190/j.jbupt.2019-103

北京邮电大学学报 ›› 2020, Vol. 43 ›› Issue (2): 87-93.doi: 10.13190/j.jbupt.2019-103

基于深度强化学习的综合能源业务通道优化机制

马庆刘¹, 喻鹏¹, 吴佳慧¹, 熊翱¹, 颜拥²

1. 北京邮电大学网络与交换技术国家重点实验室, 北京 100876;
2. 国网浙江省电力有限公司, 杭州 310007

收稿日期:2019-05-31 发布日期:2020-04-28
通讯作者: 喻鹏(1986-),男,副教授,E-mail:yupeng@bupt.edu.cn. E-mail:yupeng@bupt.edu.cn
作者简介:马庆刘(1994-),男,硕士生.
基金资助:
国家电网公司科技项目"高可信智能感知互动综合服务系统关键技术研发及应用示范"（52110418002V）

A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning

MA Qing-liu¹, YU Peng¹, WU Jia-hui¹, XIONG Ao¹, YAN Yong²

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. State Grid Zhejiang Electric Power Company Limited, Hangzhou 310007, China

Received:2019-05-31 Published:2020-04-28

摘要/Abstract

摘要： 为了保障综合能源系统的稳定运行，承载综合能源业务的通信网络需要具备高可靠、低风险等特征.依据综合能源业务的通道要求，提出了一种深度强化学习的算法，旨在对大规模综合能源业务在承载的电力通信网上寻找到整体最优的路径.该方法以整体时延和网络负载均衡度为目标，对网络拓扑进行训练，并保存模型，然后通过迭代学习获取最优的结果.仿真结果表明，该方法找到的路径既可以保证整体时延较短，又可以保证网络的整体负载均衡.同时，在网络规模很大、业务数量很多的情况下，深度强化学习算法可有效提高计算效率.

关键词: 深度强化学习, 路径优化, 时延, 负载均衡

Abstract: In order to ensure the stable operation of the integrated energy system, the integrated energy service needs to have high reliability and low risk when being carried by the communication network. According to the channel requirements of the integrated energy service, an algorithm of deep reinforcement learning is proposed, aiming to find the overall optimal path for the large-scale integrated energy service on the carried power communication network. The method that aims at the overall delay and network load balance, trains the network topology and saves the model, and then obtains the optimal result through iterative learning. The simulation results show that the routing found by this method can ensure the overall delay is short and guarantee the overall load balance of the network. At the same time, for scenarios with a large network size and a large number of services, the deep reinforcement learning algorithm can effectively improve the computational efficiency.

Key words: deep reinforcement learning, routing optimization, time delay, load balancing

中图分类号:

TN929.11

马庆刘, 喻鹏, 吴佳慧, 熊翱, 颜拥. 基于深度强化学习的综合能源业务通道优化机制[J]. 北京邮电大学学报, 2020, 43(2): 87-93.

MA Qing-liu, YU Peng, WU Jia-hui, XIONG Ao, YAN Yong. A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning[J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 87-93.

参考文献

[1] 余晓丹, 徐宪东, 陈硕翼, 等. 综合能源系统与能源互联网简述[J]. 电工技术学报, 2016, 31(1):1-13. Yu Xiaodan, Xu Xiandong, Chen Shuoyi, et al. A brief review to integrated energy system and energy Internet[J]. Transactions of China Electrotechnical Society, 2016, 31(1):1-13.
[2] 贾宏杰, 王丹, 徐宪东, 等. 区域综合能源系统若干问题研究[J]. 电力系统自动化, 2015, 39(7):198-207. Jia Hongjie, Wang Dan, Xu Xiandong, et al. Research on some key problems related to integrated energy systems[J]. Automation of Electric Power Systems, 2015, 39(7):198-207.
[3] 蔡伟, 杨洪, 熊飞, 等. 考虑电力通信网可靠性的业务路由优化分配方法[J]. 电网技术, 2013, 37(12):3541-3545. Cai Wei, Yang Hong, Xiong Fei, et al. An optimized service routing allocation method for electric power communication network considering reliability[J]. Power System Technology, 2013, 37(12):3541-3545.
[4] 王浩, 李知航, 潘志文, 等. LTE网络中具备QoS保障的动态负载均衡算法[J]. 中国科学:信息科学, 2012, 42(6):674-686. Wang Hao, Li Zhihang, Pan Zhiwen, et al. QoS guaranteed dynamic load balancing algorithm in 3GPP LTE networks[J]. Scientia Sinica(Informationis), 2012, 42(6):674-686.
[5] 高钧利. SDH光传输系统的时延测算[J]. 浙江电力, 2011, 30(4):42-45. Gao Junli. Time delay test and calculation in SDH, based optical transmission system[J]. Zhejiang Electric Power, 2011, 30(4):42-45.
[6] 周泰. 图的深度优先遍历算法及运用[J]. 电脑编程技巧与维护, 2011(16):93-94. Zhou Tai. The DFS for graph and its application[J]. Computer Programming Skills & Maintenance, 2011(16):93-94.
[7] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1):86-100. Gao Yang, Chen Shifu, Lu Xin. Research on reinforcement learning technology:a review[J]. Acta Automatica Sinica, 2004, 30(1):86-100.
[8] Mnih V, Kavukcuoglu K, Silver D, et al. Human, level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[9] 刘建伟, 高峰, 罗雄麟. 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019, 42(06):1406-1438. Liu Jianwei, Gao Feng, Luo Xionglin. Survey of deep reinforcement learning based on value function and policy gradient[J]. Chinese Journal of Computers, 2019, 42(06):1406-1438.

[1]	刘阳, 滕颖蕾, 牛涛, 郅佳琳. 基于深度强化学习的滤波器剪枝方案[J]. 北京邮电大学学报, 2023, 46(3): 31-36.
[2]	谭炜骞, 吴斌伟, 汪硕. 确定性网络跨域传输架构与DRL流量调度算法[J]. 北京邮电大学学报, 2023, 46(3): 37-42.
[3]	杨华, 耿烜, 孔宁. 一种采用dueling-DDQN算法的无线网络MAC协议[J]. 北京邮电大学学报, 2023, 46(3): 25-30.
[4]	公雨魏翼飞. 一种集成学习辅助DDPG的资源优化算法[J]. 北京邮电大学学报, 2023, 46(2): 29-36.
[5]	孙国玮许方敏朱瑾瑜张恒升赵成林. 算力网络中的确定性调度与路由联合智能优化方案[J]. 北京邮电大学学报, 2023, 46(2): 9-14.
[6]	郭兴康孙君. 基于交替方向乘子法与深度强化学习算法的资源分配[J]. 北京邮电大学学报, 2022, 45(6): 126-130.
[7]	郭令奇褚智贤廖建新王敬宇陆璐. 意图驱动的自智网络资源按需服务[J]. 北京邮电大学学报, 2022, 45(6): 85-91.
[8]	韩书君吕素玉许晓东陶小峰韩伯骁. 智能反射面辅助的短包通信系统：时延与安全性能分析[J]. 北京邮电大学学报, 2022, 45(6): 70-77.
[9]	苏扬, 魏连锁, 郭媛. 多目标融合势博弈无线自组网络拓扑控制算法[J]. 北京邮电大学学报, 2022, 45(4): 91-97.
[10]	苏扬魏连锁郭媛. 多目标融合的序数势博弈无线自组网络拓扑控制算法[J]. 北京邮电大学学报, 2022, 45(4): 105-111.
[11]	郅佳琳, 王楠, 满毅, 滕颖蕾. 面向硬件感知的边缘计算卸载和资源分配[J]. 北京邮电大学学报, 2022, 45(2): 22-28.
[12]	高月红, 杨昊天, 陈露, 杨鸿文, 尹宁. 基于模糊逻辑的eMBB/URLLC复用机制选择算法[J]. 北京邮电大学学报, 2021, 44(3): 15-20,34.
[13]	任首首, 刘冰洋, 王闯, 孟锐, 刘轩. 基于网络演算理论的Damper调度机制[J]. 北京邮电大学学报, 2021, 44(2): 26-32.
[14]	杨鹏, 张义富, 李职杜, 吴大鹏, 王汝言. 面向可靠端边协同的时延保障模型[J]. 北京邮电大学学报, 2021, 44(2): 47-53.
[15]	康曼聪, 李曦, 纪红, 张鹤立. 基于任务间依赖关系的小小区协作卸载策略[J]. 北京邮电大学学报, 2021, 44(1): 72-78.

基于深度强化学习的综合能源业务通道优化机制

A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价