您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（理学版）》

山东大学学报（理学版） ›› 2017, Vol. 52 ›› Issue (5): 104-110.doi: 10.6040/j.issn.1671-9352.2.2016.217

• • 上一篇    

面向云平台的异构监控数据存储方法

鞠瑞1,2,王丽娜1,2,余荣威1*,徐来1   

  1. 1.武汉大学计算机学院, 湖北 武汉 430072;2.武汉大学空天信息安全与可信计算教育部重点实验室, 湖北 武汉 430072
  • 收稿日期:2016-09-02 出版日期:2017-05-20 发布日期:2017-05-15
  • 通讯作者: 余荣威(1981— ),男,博士研究生,副教授,研究方向为可信计算、密码协议. E-mail:roewe.yu@whu.edu.cn E-mail:1584140885@qq.com
  • 作者简介:鞠瑞(1991— ),女,硕士研究生,研究方向为虚拟机监控、数据存储. E-mail:1584140885@qq.com
  • 基金资助:
    国家自然科学基金资助项目(U1536204,61373169);国家科技支撑计划项目(2014BAH41B00);国家高技术研究发展计划(863计划)项目(2015AA016004);信息保障技术重点实验室开放基金(KJ-14-110,KJ-14-101)

A method of storing heterogeneous monitoring data for cloud platform

JU Rui1,2, WANG Li-na1,2, YU Rong-wei1*, XU Lai1   

  1. 1. Computer School, Wuhan University, Wuhan 430072, Hubei, China;
    2. Key Laboratory of Aerospace Information Security and Trusted Computing Ministry of Education, Wuhan University, Wuhan 430072, Hubei, China
  • Received:2016-09-02 Online:2017-05-20 Published:2017-05-15

PDF (PC)

605

摘要： 针对云平台监控数据异构、数据量大、冗余量大、时序性强的特点,提出一种异构监控数据存储方法。该方法以反范式化为基础,采用统一数据表达方式兼容异构源数据,通过结合标签分离、数据压缩、预热缓存、索引等机制实现了云平台监控数据的高效存储,提高了存储空间利用率,增强了数据的可扩展性,简化数据管理,降低了数据的开发维护成本。实验结果表明:本文所提出方法满足云平台监控数据高效写入、实时更新与快速查询的实际要求。

关键词: 云平台监控, 数据存储, 反范式化

Abstract: Considering that the monitoring data was heterogeneous, large in amount, time-ordered and of great redundancy; a method to store heterogeneous monitoring data was proposed. This method was based on anti-normalization. It adapted united data organization manner to make source data compatible. At the same time, through a combination of mechanisms like label separation, data compression, caching and index, it can also realized the high-efficiency storage of monitoring data, increased the utilization of storage space, enhanced the scalability of data, simplified the data management and reduced the development and maintenance cost of data. The results of experiment showed that this method could satisfy the actual application requirement of high-efficiency write, real-time update and quick query of monitoring data in cloud platform.

Key words: data storage, anti-normalization, cloud platform monitoring

中图分类号: 

  • TP274
[1] ROCHWERGER B, BEITGAND D, EPSTEIN A, et al. Reservoir-When one cloud is not enough[J]. Computer, 2011, 44(3):44-51.
[2] CHEN Min, JIN Hai, WEN Yonggang, et al. Enabling technologies for future data center networking: A primer[J]. IEEE Network, 2013, 27(4):8-15.
[3] YANG Hui, QIN Yong, FENG Gefei, et al. Online monitoring of geological CO2 storage and leakage based on wireless sensor networks[J]. IEEE Sensors Journal, 2013, 13(2):556-562.
[4] CHANG P H, WANG T P. Supporting personal mobility with integrated RFID in VoIP system[C] // Proceedings of the International Conference on New Trends in Information and Service Science. USA: IEEE, 2009: 1353-1359.
[5] CORNELIA G, ROBERT G, GEORGE P, et al. A comparative study: MongoDB vs MySQL[C] // International Conference on Engineering of Modern Electric Systems Oradea: IEEE, 11-12 June. 2015.
[6] SCHRAM A, ANDERSON K M. MySQL to NoSQL: data modeling challenges in supporting scalability[C] // Proceedings of the 3rd annual conference on Systems, Programming, Languages and Applications: Software for Humanity. New York: ACM, 2012: 191-202.
[7] 丁治明, 高需. 面向物联网海量传感器采样数据管理的数据库集群系统框架[J]. 计算机学报, 2012, 35(6):1175-1191. DING Zhiming, GAO Xu. A database cluster system framework for managing massive sensor sampling data in the internet of things[J]. Chinese Journal of Computers, 2012, 35(6):1175-1191.
[8] CAO Wei, YU Feng, XIE Jiasen. Realization of the low cost and high performance MySQL cloud database[J]. Proceedings of the Vldb Endowment, 2014, 7(13):1742-1747.
[9] Cattell, Rick. Scalable SQL and NoSQL Data Stores[J]. Acm Sigmod Record, 2010, 39(4):12-27.
[10] LI Yishan, MANOHARAN S. A performance comparison of SQL and NoSQL databases[C] //IEEE Pacific Rim Conference on Communications, Computers & Signal Processing. USA: IEEE, 2013: 15-19.
[11] STONEBRAKER M. SQL databases v. NoSQL databases[J]. Communications of the Acm, 2010, 53(4):10-11.
[12] SCHREINER GA, DUARTE D, RONALDO DSM. SQLtoKeyNoSQL: a layer forrelational to key-based NoSQL database mapping[C] // International Conference on Information Integration and Web-Based Applications & Services. New York: ACM, 2015.
[13] ANICETO R, XAVIER R, HOLANDA M, et al. Genomic data persistency on a NoSQL database system[C] // IEEE International Conference on Bioinformatics and Biomedicine. USA: IEEE, 2014: 8-14.
[14] FRANCESCO M D, LI Na, RAJ M, et al. A storage infrastructure for heterogeneous and multimedia data in the internet of things[C] // IEEE International Conference on Green Computing and Communications. USA: IEEE, 2012: 487-487.
[15] JEONG S, BYUN J, KIM D, et al. A data management infrastructure for bridge monitoring[J]. Proceedings of SPIE-The International Society for Optical Engineering, 2015, 9435.
[16] 田野, 袁博, 李廷力. 物联网海量异构数据存储与共享策略研究[J]. 电子学报, 2016, 44(2): 247-257. TIAN Ye, YUAN Bo, LI Tingli. A massive and heterogeneous data storage and sharing strategy for internet of things[J]. Chinese Journal of Electronics, 2016, 44(2):247-257.
[1] 赵官宝, 刘云. 一种基于位表的有效频繁项集挖掘算法[J]. 山东大学学报（理学版）, 2015, 50(05): 23-29.
[2] 张曰云. 随机函数逆P-集合与其属性依赖特征[J]. 山东大学学报（理学版）, 2014, 49(10): 90-94.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《山东大学学报(理学版)》编辑部　
地址: 山东省济南市山大南路27号山东大学中心校区（250100）　电话: +86-0531-88366917　E-mail: xblxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发