云计算Hadoop平台的异常数据检测算法研究

Cloud Computing Hadoop Platform of Abnormal Data Detection Algorithm Research

摘要:

近年来,随着我国互联网技术的飞速发展与大规模网络运算平台研究的深入,云平台下的数据处理已成为大规模数据的主要处理方式；但是,现有的云计算Hadoop平台在海量数据异常涌入状态下,常常出现数据逻辑错误、数据链完整性缺失、数据失效的问题,造成无法对上述异常数据进行有效检测处理,严重影响云计算Hadoop平台的数据运算准确性；针对上述问题,提出云计算Hadoop平台的异常数据检测算法研究方法；采用JNS数据采集筛查模组、算法逻辑补偿模组与动态反馈模组对现有的云端计算平台存在的问题进行针对性解决；通过仿真模拟实验证明,提出的云计算Hadoop平台的异常数据检测算法研究方法,具有异常数据识别率高,准确性高,速度快、可实施性强、稳定性好的特点。

Abstract:

In recent years, with the rapid development of Internet technology in our country development of further research and large-scale network computing platform, YunPing audience data processing, has become the main way of massive data. However, the existing cloud computing platform in abnormal huge amounts of data into state, often appear logical error, loss of data link integrity, data, solve the problem of a can to effectively detect the abnormal data processing, seriously affect the accuracy of cloud computing platform data operation. According to the above problem, a Hadoop cloud computing platform of abnormal data detection algorithm research methods. By screening JNS data collection module, the algorithm logic compensation module and dynamic feedback module to the existing problems of cloud computing platform, targeted to solve. Through the simulation experiment proves that the Hadoop cloud computing platform of abnormal data detection algorithm research method, has high recognition rate anomaly data, high accuracy, fast speed, strong practical, good stability characteristics.