关键词: 时间敏感网络, 算力网络, 深度强化学习

Abstract: The Compute first network (CFN) integrates heterogeneous computing power information with the network to improve resource utilization and network transmission efficiency. The time-sensitive network (TSN) ensures low-latency and high-reliability transmission performance. The fusion of the two can achieve high efficiency deterministic forwarding. The resource scheduling and routing planning in the integrated decision-making CFN and the gate control arrangement in the TSN will have problems such as too many decision variables, too high computational complexity, and insufficient optimization performance. In response to the above problems, a fusion architecture based on IEEE 802.1Qbv for gated arrangement, computing network routing planning, and computing resource scheduling is proposed. Based on deep reinforcement learning, an improved RBDQN (reward-back deep Q-learning) algorithm is proposed to optimize gate control list, and a greedy algorithm is used to assist routing path planning. The algorithm establishes a utility function based on the average delay, energy consumption and user satisfaction as multiple optimization indicators. The simulation results show that, compared with the genetic algorithm, RBDQN can reduce the convergence time of small-scale scheduling problems by more than 1 times, and can reduce the convergence time by dozens of times for multi-service and multi-node computing network problems. At the same time, the algorithm can avoid the model from falling into a local optimum. Compared with the traditional DQN, the decision result improves the performance of the utility function index by more than 10%, and the convergence time under the same index decreases by about 50%.

Key words: time sensitive network, computing first network, deep reinforcement learning