Journal of Beijing University of Posts and Telecommunications

  • EI核心期刊

Journal of Beijing University of Posts and Telecommunications ›› 2021, Vol. 44 ›› Issue (5): 127-132.doi: 10.13190/j.jbupt.2021-017

• REPORTS • Previous Articles    

Click-Through Rate Prediction of Multi-Head Self-Attention in Hyperbolic Space

HAN Yue-lin, WANG Xiao-yu   

  1. School of Computer Science and Technology, Harbin University of Science and Technology, Harbin 150080, China
  • Received:2021-01-27 Online:2021-10-28 Published:2021-09-06

Abstract: In recommendation systems,understanding the complex functional interactions behind user behaviors is crucial to predict the clicking probability of users on advertisements or commodities. Efforts have been made to find low-dimensional representations and meaningful combinations of sparse and high-dimensional original features. Among them,the deep & cross network can explicitly cross features at each layer. However it treats all crossing features "equally" and does not consider the influence of different features on the results,which may eliminate some useful information. Therefore,a prediction model of click-through rate of multi-head self-attention neural network in hyperbolic space is proposed. In hyperbolic space,the model uses Lorentzian distance instead of inner product, to measure the similarity and correlation between features, which can avoid dimension disaster. Experimental results show that the model is superior to the deep & cross network on predicting click-through rate data sets in terms of accuracy.

Key words: hyperbolic space, multi-head self-attention, Lorentzian distance

CLC Number: