基于云计算的大数据自动分类处理系统设计

Design of Large Data Automatic Classification and Processing System Based on Cloud Computing

摘要:

随着现代网络技术不断进步,系统数据量也在逐渐增多;传统的大数据自动分类处理系统已经无法满足现阶段用户需求,其软件与硬件的设计都比较单一,存在能源消耗大、分类速度慢、处理时间长、内存占用率高等问题,为此,提出基于云计算的大数据自动分类处理系统的设计;首先设计系统硬件结构,主要包括数据采集器、数据处理器以及数据自动存储模块,并详细的介绍了各硬件结构；然后利用时域特征提取数据的算法对频域特征数据进行提取,从而实现数据自动分类处理系统的软件设计；最后对两种系统性能进行对比实验;实验结果证明,基于云计算的大数据自动分类处理系统的资源不仅占用率低,内存消耗小,而且数据库内存较大;该系统不但可以提高数据自动分类精准度,还能加快数据分类速度,从而使系统拥有更好的分类性能。

Abstract:

With the continuous improvement of modern network technology, the amount of data in the system is increasing gradually. Traditional big data automatic classification processing system has been unable to meet the needs of users, the software and hardware design are single, there exists large energy consumption, slow speed of classification, long processing time and memory usage rate is high, therefore, automatic classification is proposed based on cloud computing of large data processing system design. Firstly, the hardware structure of the system is designed, which mainly includes data collector, data processor and data automatic storage module, and introduces the structure of each hardware in detail. Then, the data is extracted using the time-domain feature extraction algorithm to realize the software design of data automatic classification and processing. Finally, two kinds of system performance design are compared. The results show that the resources of large data automatic classification and processing system based on cloud computing have low occupancy rate, small memory consumption and large memory of database. The design of this system can not only improve the accuracy of automatic classification of data, but also speed up the classification of data, so that the system has better classification performance.