基于互相关分析和SOM神经网络的异常值检测平台

An Outlier Detection Platform Based on Cross-Correlation Analysis and SOM Neural Network

摘要:

异常值的检测问题是时下数据挖掘领域的研究热点。目前已经有许多种成熟的异常值检测方法,但当数据是高维混合型属性,或者存在成片孤立点时,这些方法就变得很不理想甚至不再适用。因此,针对这些现有方法的不足之处,提出了新的孤立点检测方法,并设计了时域和空域的异常值检测平台。对于时间和空间序列数据集,该平台分别采用基于互相关分析和自组织竞争(self-organizing maps, SOM)神经网络的异常值检测方法。经实验验证,检测平台具有较高的检测率和可靠性。同时,在搭建该平台时充分考虑了模块化和层次化的方式,使得平台具有良好的可扩展性和开放性。

Abstract:

The outlier detection problem has become the focus of research in the field of data mining. At present, there are many kinds of mature outlier detection methods. But when the data has high dimensional mixed property, or there are assembled outliers, the result of these methods become unsatisfactory or imapplicable. Therefore, in view of the shortcomings of these existing methods, a new outlier detection method is proposed in this paper. Meanwhile, we designed the outlier detection platform in time and space domain. For time and spatial series datasets, the platform is based on cross-correlation analysis and self-organizing maps (SOM) neural network clustering. It can be proved by experiments that the platform has higher detection rate and reliability. By the way, the platform is built in a modular and hierarchical way with good openness and extensibility.