大数据预处理中属性约简的特性保持分析

Characteristics analysis of attribute reduction in big data preprocessing

摘要:

伴随着工业及社会信息化程度的增强,各个领域的自动化程度越来越高,大规模及超大规模海量数据应运而生,呈现出大数据特征。这些海量数据在提升行业发展动力的同时,也带来了巨大的挑战性问题—数据可用性。为了从海量数据中甄别出无用信息、挖掘有利于相关领域发展的有价值信息,就需要对其进行数据分析。数据预处理技术可以极大地减少数据分析时的处理量,提高数据分析处理的效率,而属性约简在数据预处理中是一个比较重要的环节。在分析大数据属性特征的基础上,较系统地分析了目前几种主流的属性约简算法,对各类算法的性能进行了剖析,并展望了今后大数据预处理的研究工作思路。

Abstract:

Along with the enhancement of the degree of industry and social information, the degree of automation in every field is higher and higher, and the massive data of large scale and super large scale emerge as the times require and present the features of big data.SThese massive amounts of data in the promotion of industry development momentum play an important role, at the same time, brings a huge challenge - data usability issues.SIn order to identify the invaluable information and dig out the valuable information benefit to the development of related fields from the massive data, it is necessary to analyze the data. Data preprocessing technique can greatly reduce the amount of analysis data and improve the processing efficiency of data analysis, and attribute reduction in data preprocessing is an important link. On the basis of analyzing the properties of large data, systematically analyzes the current several mainstream attribute reduction algorithm, and analyzes the performance of each algorithm, then, according to the results of the comparison, forecast the future research work in the field of attribute reduction.