Traffic Distribution Algorithm Based on Multi-Agent Reinforcement Learning

doi:10.13190/j.jbupt.2019-140

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2019, Vol. 42 ›› Issue (6): 43-48,57.doi: 10.13190/j.jbupt.2019-140

• Papers • Previous Articles Next Articles

Traffic Distribution Algorithm Based on Multi-Agent Reinforcement Learning

CHENG Chao¹, TENG Jun-jie², ZHAO Yan-ling³, SONG Mei¹

1. School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. China Financial Certification Authority, Beijing 100054, China;
3. Instrumentation Technology and Economy Institute, Beijing 100055, China

Received:2019-07-10 Online:2019-12-28 Published:2019-11-15
Supported by:

Abstract

Abstract: Most of the researches on traditional traffic engineering strategies focus on constructing and solving mathematical models. To reduce computational complexity,an experience-driven traffic allocation algorithm based on multi-agent reinforcement learning was proposed. It can effectively distribute traffic on pre-calculated paths without solving complex mathematical models and then fully utilize network resources. The algorithm performs centralized training on the software defined networking controller,and can be executed on the access switch or router in a distributed way after the training is completed. Frequent interactions with the controller are avoided at the same time. Experiments show that the algorithm is effective in reducing the end-to-end delay and increasing throughput of the network with respect to the shortest-path and the equal-cost multi-path.

Key words: traffic engineering, multi-agent reinforcement learning, software-defined networking, delay, throughput

CLC Number:

TP393.0

CHENG Chao, TENG Jun-jie, ZHAO Yan-ling, SONG Mei. Traffic Distribution Algorithm Based on Multi-Agent Reinforcement Learning[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(6): 43-48,57.

References

[1] Agarwal S, Kodialam M, Lakshman T V. Traffic engineering in software defined networks[C]//2013 Proceedings IEEE INFOCOM. Turin, Italy:IEEE Press, 2013:2211-2219.
[2] Silver D, Huang A, Maddison C J, et al. Mastering the game of go with deep neural networks and tree search[J]. Nature, 2016, 529(7587):484-489.
[3] Mestres A, Hibbett M J, Estrada G, et al. Knowledge-defined networking[J]. ACM SIGCOMM Computer Communication Review, 2017, 47(3):2-10.
[4] Chavula J, Densmore M, Suleman H. Using SDN and reinforcement learning for traffic engineering in UbuntuNet Alliance[C]//2016 International Conference on Advances in Computing and Communication Engineering (ICACCE). Durban, South Africa:IEEE Press, 2016:349-355.
[5] Xu Zhiyuan, Tang Jian, Meng Jingsong, et al. Experience-driven networking:a deep reinforcement learning based approach[C]//IEEE INFOCOM 2018-IEEE Conference on Computer Communications. Honolulu, USA:IEEE Press, 2018:1871-1879.
[6] Sutton R S, Barto A G. Reinforcement learning:an introduction[M]. Cambrige:MIT Press, 1998.
[7] Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning[EB/OL]. 2013(2013-12-19)[2019-07-05]. https://arxiv.org/pdf/1312.5602.pdf.
[8] Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[C]//Advances in Neural Information Processing Systems. Long Beach, USA:Curran Associates, 2017:486-496.
[9] Lowe R, Wu Y, Tamar A, et al. Multi-agent-actor-critic for mixed cooperative-competitive environments[C]//Advances in Neural Information Processing Systems. Long Beach, USA:Curran Associates, 2017:6379-6390.
[10] Winstein K, Balakrishnan H. Tcp ex machina:Computer-generated congestion control[C]//ACM SIGCOMM 2013. Hong Kong, China:ACM Press, 2013:123-134.
[11] 张峰, 李刚, 宋丽. 一种适应网络拥塞的网络端到端时延估算模型[J]. 空军雷达学院学报, 2009, 23(3):190-193. Zhang Feng, Li Gang, Song Li. An estimation model of end-to-end delay of network congestion[J]. Journal of Air Force Radar Academy, 2009, 23(3):190-193.

[1]	YANG Hua, GENG Xuan, KONG Ning. A MAC Protocol for Wireless Networks Using Dueling-DDQN Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 25-30.
[2]	. RIS Assisted Short Packet Communication System: Delay and Security Performance Analysis [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(6): 70-77.
[3]	DU Mei, ZHOU Junhua, LI Dunqiao, CHEN Shizhao, WEI Yifei. A Joint Intelligent Optimization Scheme of Computation Offloading and Resource Allocation for MEC [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 65-71.
[4]	WEN Wei. Routing Algorithm Based on Density Clustering for Delay Tolerant Network [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 137-142.
[5]	WANG Ming-wei, LI Hui-zhen. Multi-Slot Energy Harvesting Wireless Communication over Nakagami-m Channel Fading [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(4): 54-60.
[6]	GUO Shao-xiong, LIU Yu-tao, Lü Yu-jing, ZHANG Zhong-zhao. Resource Allocation Algorithm for Simultaneous Wireless Information and Power Transfer in Multi-Carrier Cognitive Radio [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(3): 19-23.
[7]	XU Meng-meng, ZHU Hai, CUI Ya-jie, XU Heng-zhou. Energy-Efficient Routing with Delay Soft-Constraint in Sparse Mobile Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(3): 72-76.
[8]	MA Qing-liu, YU Peng, WU Jia-hui, XIONG Ao, YAN Yong. A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 87-93.
[9]	MENG Hao, HUO Ru, GUO Qian-ying, HUANG Tao, LIU Yun-jie. Machine Learning-Based Stochastic Task Offloading Algorithm in Mobile-Edge Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(2): 25-30.
[10]	ZHANG Xin-ran, SUN Song-lin. Policy Estimation Error Analysis for Symmetrical MARL Problem in Communication Resource Scheduling [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(2): 1-6.
[11]	TANG Lun, YANG Heng, ZHAO Guo-fan, WANG Yao-wei, CHEN Qian-bin. Delay-Aware 5G Network Slicing Node and Link Embedding Algorithm [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(6): 71-77.
[12]	TANG Lun, ZHAO Pei-pei, ZHAO Guo-fan, CHEN Qian-bin. Dynamic Deployment Algorithm for Service Function Chaining with QoS Guarantee [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(6): 90-96.
[13]	LI Min-zheng, LI Yan-feng. Research on Downlink Energy Efficiency of Multi-Cell Massive Distributed Antenna System [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(6): 26-33.
[14]	GAO Yue-hong, LI Lan, ZHANG Xin, YANG Hong-wen. Analysis and Research on the Probabilistic Delay Constraint and Resource Reservation for URLLC Traffic [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(5): 98-102.
[15]	GUO Bo-ren, ZHANG Xin, YANG Hong-wen. A Random Access Scheme Based on Sub-Preambles for IoT Systems [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(5): 110-114.

Traffic Distribution Algorithm Based on Multi-Agent Reinforcement Learning

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments