高速网络流实时处理模型研究与实现

doi:10.6040/j.issn.1671-9352.2.2021.123

《山东大学学报(理学版)》 ›› 2022, Vol. 57 ›› Issue (9): 25-32.doi: 10.6040/j.issn.1671-9352.2.2021.123

高速网络流实时处理模型研究与实现

苏自鹏^1,2,袁磊³,刘鹏³,陈兴蜀^1,2*,罗永刚^1,2,陈良国^1,2

1.四川大学网络空间安全学院, 四川成都 610207;2.四川大学网络空间安全研究院, 四川成都 610207;3.中移(成都)信息通信科技有限公司, 四川成都 611730

发布日期:2022-09-15
作者简介:苏自鹏(1996— ),男,硕士研究生,研究方向为云计算与大数据安全. E-mail:suzipeng@stu.scu.edu.cn*通信作者简介:陈兴蜀(1968— ),女,博士,教授,博士生导师,研究方向为云计算、信息安全、计算机网络. E-mail:chenxsh@scu.edu.cn
基金资助:
中国移动科研基金(CM20200409)

Research and implementation of real-time processing model of high-speed network stream

SU Zi-peng^1,2, YUAN Lei³, LIU Peng³, CHEN Xing-shu^1,2*, LUO Yong-gang^1,2, CHEN Liang-guo^1,2

1. School of Cyber Science and Engineering, Sichuan University, Chengdu 610207, Sichuan, China;
2. Cybersecurity Research Institute, Sichuan University, Chengdu 610207, Sichuan, China;
3. China Mobile(Chengdu)Information Communication Technology Co., Ltd., Chengdu 611730, Sichuan, China

Published:2022-09-15

摘要/Abstract

摘要： 针对当前网络流处理存在的实时性不足,难以适应不同网络流量规模的问题,研究并实现了一种多流量场景实时网络流重组架构,旨在进行网络流处理时能够适应不同的网络流量场景,并且有效提升系统资源利用率。该架构基于DPDK实现高速流量实时采集,并且对流重组过程进行阶段划分,构造不同场景的运行拓扑,实现系统资源的动态分配。在单机环境下实现了该系统,并在10 Gbps网络环境下进行测试,结果表明该架构在吞吐量和资源利用率上优于同类其他方法。

关键词: 网络安全, 流记录, 网络流特征提取, DPDK

Abstract: With the increasing scale of network traffic, it becomes more and more difficult to analyze network traffic in real time, efficiently and accurately. Aiming at the problem that the current network flow processing is not real-time enough and it is difficult to adapt to different network traffic scale, this paper studies and implements a real-time network flow restructuring architecture with multiple traffic scenarios,Designed to be able to adapt to different network traffic scenarios during network stream processing, And effectively improve system resource utilization. The architecture is based on DPDK to achieve real-time collection of high-speed traffic, and the process of stream reorganization is divided into stages, to construct operating topologies for different scenarios, and to achieve dynamic allocation of system resources. The system was implemented in a stand-alone environment and tested in a 10Gbps network environment. The results show that the architecture is superior to similar methods in throughput and resource utilization.

Key words: network security, flow record, network flow feature extraction, DPDK

中图分类号:

TP393

苏自鹏,袁磊,刘鹏,陈兴蜀,罗永刚,陈良国. 高速网络流实时处理模型研究与实现[J]. 《山东大学学报(理学版)》, 2022, 57(9): 25-32.

SU Zi-peng, YUAN Lei, LIU Peng, CHEN Xing-shu, LUO Yong-gang, CHEN Liang-guo. Research and implementation of real-time processing model of high-speed network stream[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(9): 25-32.

参考文献

[1] GU G, PORRAS P A, YEGNESWARAN V, et al. Bothunter: detecting malware infection through ids-driven dialog correlation[C] //Proceedings of the 16th USENIX Security Symposium. Boston: USENIX, 2007, 7: 1-16.
[2] KARAGIANNIS T, BROIDO A, FALOUTSOS M, et al. Transport layer identification of P2P traffic[C] //Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement. Taormin: ACM, 2004: 121-134.
[3] PAPADOGIANNAKIS A, POLYCHRONAKIS M, MARKATOS E P. Stream-oriented network traffic capture and analysis for high-speed networks[J]. IEEE Journal on Selected Areas in Communications, 2014, 32(10): 1849-1863.
[4] DERI L, SPA N E T. nProbe: an open source netflow probe for gigabit networks[C] //TERENA Networking Conference. Zagreb: CARNet, 2003: 1-4.
[5] INACIO C M, TRAMMELL B. Yaf: yet another flowmeter[C] //Proceedings of LISA10: 24th Large Installation System Administration Conference. San Diego: SEI, 2010: 107.
[6] CLAISE B. Cisco systems NetFlow services export version 9[EB/OL].[2021-06-06]. RFC 3954, 2004.4. http://www.doc88.com/p-9502720205086.html.
[7] WANG M, LI B, LI Z. sFlow: towards resource-efficient and agile service federation in service overlay networks[C] //24th International Conference on Distributed Computing Systems, 2004. Proceedings. Tokyo:IEEE, 2004: 628-635.
[8] SANTIAGO DEL RIO P M, ROSSI D, GRINGOLI F, et al. Wire-speed statistical classification of network traffic on commodity hardware[C] //Proceedings of the 2012 Internet Measurement Conference. Boston: ACM, 2012: 65-72.
[9] ZHANG T, LINGUAGLOSSA L, GALLO M, et al. FloWatcher-DPDK: lightweight line-rate flow-level monitoring in software[J]. IEEE Transactions on Network and Service Management, 2019, 16(3): 1143-1156.
[10] EMMERICH P, PUDELKO M, GALLENMÜLLER S, et al. Flowscope: efficient packet capture and storage in 100 gbit/s networks[C] //2017 IFIP Networking Conference(IFIP Networking)and Workshops. Stockholm: IEEE, 2017: 1-9.
[11] EMMERICH P, PUDELKO M, SCHEITLE Q, et al. Efficient dynamic flow tracking for packet analyzers[C] //2018 IEEE 7th International Conference on Cloud Networking(CloudNet). Tokyo: IEEE, 2018: 1-6.
[12] TREVISAN M, FINAMORE A, MELLIA M, et al. Traffic analysis with off-the-shelf hardware: challenges and lessons learned[J]. IEEE Communications Magazine, 2017, 55(3): 163-169.
[13] APACHE Metron. Apache metron home page[EB/OL]. [2020-03-20]. http://metron.apache.org/.
[14] 王煜骢,陈兴蜀,罗永刚,等.NTCI-Flow:一种可扩展的高速网络流量处理框架[J].工程科学与技术,2017,49(S1):168-174. WANG Yucong, CHEN Xingshu, LUO Yonggang, et al. NTCI-Flow: an extensible framework for high speed network traffic processing[J]. Advanced Engineering Sciences, 2017, 49(S1):168-174.
[15] WANG M, LIU J, ZHOU W. Design and implementation of a high-performance stream-oriented big data processing system[C] //2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics(IHMSC). Hangzhou: IEEE, 2016, 1: 363-368.
[16] CAI Y, WU B, ZHANG X, et al. Flow identification and characteristics mining from internet traffic with hadoop[C] //2014 International Conference on Computer, Information and Telecommunication Systems(CITS). Jeju: IEEE, 2014: 1-5.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

高速网络流实时处理模型研究与实现

Research and implementation of real-time processing model of high-speed network stream

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

多维度评价

本文评价

推荐阅读 10

[1]	崔朝阳,孙甲琦,徐松艳,蒋鑫. 适用于集群无人机的自组网安全分簇算法[J]. 山东大学学报（理学版）, 2018, 53(7): 51-59.
[2]	朱丹,谢晓尧,徐洋,夏梦婷. 基于云模型与贝叶斯反馈的网络安全等级评估方法[J]. 山东大学学报（理学版）, 2018, 53(1): 53-62.
[3]	李阳,程雄,童言,陈伟,秦涛,张剑,徐明迪. 基于流量统计特征的潜在威胁用户挖掘方法[J]. 山东大学学报（理学版）, 2018, 53(1): 83-88.
[4]	吴欢,詹静,赵勇,陶政,杨静. 一种高效虚拟化多级网络安全互联机制[J]. 山东大学学报（理学版）, 2016, 51(3): 98-103.
[5]	吕良杨波陈贞翔. 网络安全防护系统的研究与设计[J]. J4, 2009, 44(9): 47-51.
[6]	张甲,段海新,葛连升 . 基于事件序列的蠕虫网络行为分析算法[J]. J4, 2007, 42(9): 36-40 .