Policy Estimation Error Analysis for Symmetrical MARL Problem in Communication Resource Scheduling

doi:10.13190/j.jbupt.2018-121

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM

• Papers • Next Articles

Policy Estimation Error Analysis for Symmetrical MARL Problem in Communication Resource Scheduling

ZHANG Xin-ran, SUN Song-lin

1. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Key Laboratory of Trustworthy Distributed Computing and Service(Ministry of Education), Beijing University of Posts and Telecommunications, Beijing 100876, China;
3. National Engineering Laboratory for Mobile Network Security, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2018-06-20 Online:2019-04-28 Published:2019-04-28

Abstract

Abstract: Considering multi-agent reinforcement learning (MARL) theory in communication resource scheduling scenario, the symmetrical MARL problem was proposed with definitions for three types of symmetry properties and analysis of policy estimation error. The policy estimation error theorem for strong symmetrical MARL was presented. Simulation results based on the admission control problem in wireless system were modeled by MARL, which testify the characteristics of policy estimation error for strong symmetrical MARL problems. It shows that using the MARL sub-problems with low computational complexity to estimate the original MARL problem with high computational complexity only brings small policy estimation error and deterioration of system performance.

Key words: reinforcement learning, symmetrical multi-agent reinforcement learning, policy estimation

CLC Number:

TN929.53

ZHANG Xin-ran, SUN Song-lin. Policy Estimation Error Analysis for Symmetrical MARL Problem in Communication Resource Scheduling[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, doi: 10.13190/j.jbupt.2018-121.

[1]	LIU Yang, TENG Yinglei, NIU Tao, ZHI Jialin. Filter Pruning Algorithm Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 31-36.
[2]	TAN Weiqian, WU Binwei, WANG Shuo. Cross-Domain Deterministic Networking Architecture and DRL Flow Scheduling [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 37-42.
[3]	YANG Hua, GENG Xuan, KONG Ning. A MAC Protocol for Wireless Networks Using Dueling-DDQN Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 25-30.
[4]	. An Ensemble Learning-Aided DDPG Resource Optimization Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(2): 29-36.
[5]	. Deterministic Scheduling and Routing Joint Intelligent Optimization Scheme in Computing First Network [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(2): 9-14.
[6]	. Resource Allocation Based on Alternating Direction Multiplier Method and Deep Reinforcement Learning Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(6): 126-130.
[7]	. Intent-driven Demand-aware Resource Service in Autonomous Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(6): 85-91.
[8]	CHEN Chunlei, LIU Kaijun, DONG Chen, ZHOU Hongyuan. An Adaptive Bit Rate Algorithm Model Based on Wireless Network [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(5): 115-120.
[9]	ZHI Jialin, WANG Nan, MAN Yi, TENG Yinglei. Hardware Model-Aware Joint Offloading and Resources Allocation Optimization [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 22-28.
[10]	DU Mei, ZHOU Junhua, LI Dunqiao, CHEN Shizhao, WEI Yifei. A Joint Intelligent Optimization Scheme of Computation Offloading and Resource Allocation for MEC [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 65-71.
[11]	YANG Can, LUO Tao, LIU Ying, LI Zexu, XU Yongqing. QoS Routing Algorithm in Clustered Self-Organizing Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(1): 1-6.
[12]	HUANG Hao, HU Zhi-qun, WANG Lu-han, LU Zhao-ming, WEN Xiang-ming. Intelligent Traffic Signal Control Algorithm Based on Sumtree DDPG [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(1): 97-103.
[13]	GUAN Wan-qing, ZHANG Hai-jun, LU Zhao-ming. Intelligent Resource Allocation Algorithm for 6G Multi-Tenant Network Slicing Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(6): 132-139.
[14]	MA Qing-liu, YU Peng, WU Jia-hui, XIONG Ao, YAN Yong. A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 87-93.
[15]	LIU Jin-hua, KE Zhong-ming, ZHOU Wen-hui. Reinforcement Learning Based Energy Dispatch Strategy and Control Optimization of Microgrid [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(1): 28-34.

Policy Estimation Error Analysis for Symmetrical MARL Problem in Communication Resource Scheduling

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments