网络计算机模型下海量大数据存储系统设计

Under the network computer model mass big data storage system design

摘要:

对网络计算机模型下海量大数据进行安全稳定的存储,可以提高网络计算机的使用价值,增加其使用周期。但目前的海量大数据存储方法在存储过程中,无法做到对其进行灵活高效的存储,存在大数据存储分布密度较低,存储开销较大等问题。为此,以网络计算机模型体系结构为基础,提出了一种基于ARM的海量大数据存储系统设计方法。该设计方法先利用ARM芯片对网络计算机模型下海量大数据存储系统进行硬件构造,将网络海量大数据中的可利用与不可利用数据进行分类处理,采用VISA结构根据数据分类结果对大数据存储系统软件部分进行设计,依据大数据调度模型和存储相似度特征对大数据存储的时间,质量等方面进行计算,利用计算结果对大数据传输的阈值以及分布密度进行记录,最后根据循环分段的计算方式进行冗余大数据特性压缩,并对海量大数据的常规数据和冗余数据进行存储。实验仿真证明,所提方法提高了海量大数据存储的兼容性,增强了大数据存储的精确性和灵活性。

Abstract:

The mass under the network computer models to security and stability of large data storage, can improve the use value of the network computer, increase its life cycle. But the current mass big data storage method in the process of storage, cannot afford to be flexible and efficient storage, there are large data storage distribution density is low, the problem such as storage overhead. To this end, on the basis of network computer system structure model, proposed a mass with large data storage system design method based on the ARM. Design method of the first use of ARM chips for mass big data storage system under the network computer model hardware structure, the network mass of big data available and cannot be classified by using data processing, USES the VISA structure according to the result of data classification was carried out on the big data storage system software part design, on the basis of large data scheduling model similarity characteristics of large data storage and storage time, quality, etc, to calculate, using the calculation results on the threshold of data transmission and distribution density of record, finally according to the calculation of circular section is redundant features large data compression, and the massive big data of conventional data and redundant data for storage. Experimental simulation show that the proposed method improves the compatibility of the mass big data storage, enhancing the accuracy and flexibility of the large data storage.