关键词: 广告流量, 实时, 匹配算法, 流式工作框架, 大规模数据

Abstract: A real-time internet advertisement traffic recognition system applied to massive network dataset was proposed. The model adopts the currently most popular Adblock filter rules as the basic filter rules, and combines the HashTable fast matching algorithm as well as the Aho-Corasick fast matching algorithm to recognize the advertisement traffic in a fast and real-time way. To meet the need of the massive streaming data, the algorithms are deployed on Spark Streaming, a parallel streaming framework for solving streaming data. The model is respectively experimented with both factual data from our lab and the real massive datasets from the network operators. Experiments show that the system can achieve high precision and high calculation performance.

Key words: advertisement traffic, real-time, matching algorithm, streaming framework, massive dataset