Intelligent Traffic Signal Control Algorithm Based on Sumtree DDPG

doi:10.13190/j.jbupt.2020-006

Journal of Beijing University of Posts and Telecommunications ›› 2021, Vol. 44 ›› Issue (1): 97-103.doi: 10.13190/j.jbupt.2020-006

• REPORTS • Previous Articles Next Articles

Intelligent Traffic Signal Control Algorithm Based on Sumtree DDPG

HUANG Hao^1,3,4, HU Zhi-qun², WANG Lu-han^1,3,4, LU Zhao-ming^1,3,4, WEN Xiang-ming^1,3,4

1. School of Information and Communications Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. School of Computer and Information Engineering, Hubei University, Wuhan 430062, China;
3. Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China;
4. Beijing Laboratory of Advanced Information Networks, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2020-01-17 Online:2021-02-28 Published:2021-09-30

Abstract

Abstract: A multi-intersection intelligent traffic signal control algorithm based on sumtree deep deterministic policy gradient(Sumtree DDPG)is proposed. Through real-time observation of intersection data,the cycle length,phase sequence and phase duration of the traffic signal can be intelligently adjusted to improve the efficiency of intersections. Meanwhile,the empirical data storage mode based on sumtree structure can improve the sampling efficiency and accelerate the algorithm convergence. Compared with fixed signal timing and signal timing algorithm based on traffic flow weight,a simulation is carried out that the proposed algorithm obtains good performance in vehicle queue length,vehicle waiting time and vehicle average speed in dynamic environment.

Key words: smart transportation, traffic signal control, deep reinforcement learning, deep deterministic policy gradient, multiple intersections

CLC Number:

U491.54

HUANG Hao, HU Zhi-qun, WANG Lu-han, LU Zhao-ming, WEN Xiang-ming. Intelligent Traffic Signal Control Algorithm Based on Sumtree DDPG[J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(1): 97-103.

References

[1] Araghi S, Khosravi A, Creighton D. A review on computational intelligence methods for controlling traffic signal timing[J]. Expert Systems with Applications, 2015, 42(3):1538-1550.
[2] Sims A G, Finlay A B. SCATS, splits and offsets simplified (SOS)[J]. Australian Road Research, 1984, 12(4):17-33.
[3] Lo H K. A reliability framework for traffic signal control[J]. IEEE Transactions on Intelligent Transportation Systems, 2006, 7(2):250-260.
[4] Abdulhai B, Pringle R, Karakoulas G J. Reinforcement learning for true adaptive traffic signal control[J]. Journal of Transportation Engineering, 2003, 129(3):278-285.
[5] Genders W, Razavi S. Using a deep reinforcement learning agent for traffic signal control[EB/OL]. (2016-11-03)[2020-01-10]. https://arXiv.org/abs/1611.01142.
[6] Richter S, Aberdeen D, Yu J. Natural actor-critic for road traffic optimisation[C]//NIPS 2006, Proceedings of the 19th International Conference on Neural Information Processing Systems. Vancouver:MIT Press, 2006:1169-1176.
[7] Pang Hali, Gao Weilong. Deep deterministic policy gradient for traffic signal control of single intersection[C]//2019 Chinese Control and Decision Conference (CCDC). Nanchang:IEEE, 2019:5861-5866.
[8] Casas N. Deep deterministic policy gradient for urban traffic light control[EB/OL]. (2017-08-02)[2020-01-10]. https://arXiv.org/abs/1703.09035v1.
[9] Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[C]//ICLR 2016:International Conference on Learning Representations. San Juan:[s.n.], 2016:1-14.
[10] Schaul T, Quan J, Antonoglou I, et al. Prioritized experience replay[C]//ICLR 2016:International Conference on Learning Representations. San Juan:[s.n.], 2016:1-21.
[11] Wang Xiaoqiang, Ke Liangjun, Qiao Zhimin, et al. Large-scale traffic signal control using a novel multi-agent reinforcement learning[J]. IEEE Transactions on Cybernetics, 2021, 51(1):174-187.
[12] Yang Shantian, Yang Bo, Wong Hau-San, et al. Cooperative traffic signal control using multi-step return and off-policy asynchronous advantage actor-critic graph algorithm[J]. Knowledge-Based Systems, 2019, 183:1-19.
[13] Wei Hua, Zheng Guanjie, Yao Huaxiu, et al. Intellilight:a reinforcement learning approach for intelligent traffic light control[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York:ACM, 2018:2496-2505.
[14] Zhang Zhi, Yang Jiachen, Zha Hongyuan. Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization[C]//AAMAS 2020:Proceedings of the Nineteenth International Conference on Autonomous Agents and Multi-Agent Systems. Auckland:Springer, 2020:2083-2085.

Intelligent Traffic Signal Control Algorithm Based on Sumtree DDPG

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 13

Recommended Articles

Metrics

Comments

[1]	LIU Yang, TENG Yinglei, NIU Tao, ZHI Jialin. Filter Pruning Algorithm Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 31-36.
[2]	TAN Weiqian, WU Binwei, WANG Shuo. Cross-Domain Deterministic Networking Architecture and DRL Flow Scheduling [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 37-42.
[3]	YANG Qingqing, CHEN Jian, PENG Yi. Unmanned Aerial Vehicle Trajectory Planning and Power Control Algorithm Based on Deep Deterministic Policy Gradient [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 43-48.
[4]	YANG Hua, GENG Xuan, KONG Ning. A MAC Protocol for Wireless Networks Using Dueling-DDQN Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 25-30.
[5]	. An Ensemble Learning-Aided DDPG Resource Optimization Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(2): 29-36.
[6]	. Deterministic Scheduling and Routing Joint Intelligent Optimization Scheme in Computing First Network [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(2): 9-14.
[7]	. Resource Allocation Based on Alternating Direction Multiplier Method and Deep Reinforcement Learning Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(6): 126-130.
[8]	. Intent-driven Demand-aware Resource Service in Autonomous Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(6): 85-91.
[9]	ZHI Jialin, WANG Nan, MAN Yi, TENG Yinglei. Hardware Model-Aware Joint Offloading and Resources Allocation Optimization [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 22-28.
[10]	ZHANG Tian-kui, WANG Xiao-fei, YANG Li-wei, YANG Ding-cheng. A SFC Deployment and Computation Resource Allocation Joint Algorithm in Mobile Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(1): 7-13.
[11]	GUAN Wan-qing, ZHANG Hai-jun, LU Zhao-ming. Intelligent Resource Allocation Algorithm for 6G Multi-Tenant Network Slicing Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(6): 132-139.
[12]	MA Qing-liu, YU Peng, WU Jia-hui, XIONG Ao, YAN Yong. A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 87-93.
[13]	XUE Ning, HUO Ru, ZENG Shi-qing, WANG Shuo, HUANG Tao. Tasks Offloading and Resource Scheduling Algorithm Based on Deep Reinforcement Learning in MEC [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(6): 64-69,104.