基于Hadoop的GA-BP网络在山洪预测中的研究

Study of Hadoop-based GA-BP Network in the Flash Flood Forecasting

摘要:

研究了山洪灾害监测预警系统中雨情数据的分布式存储和分布式预测；针对采集到的水文数据急剧增长和对预测精度和预报时效的要求不断提高,分别应用Hadoop分布式文件系统对数据进行分布式存储和 MapReduce框架结合遗传算法优化神经网络的权值和阈值进行分布式预测；采用基于BP神经网络的多因子山洪灾害雨量预测模型,结合遗传算法能够实现全局优化特点来优化神经网络的权值和阈值,并在数据并行处理过程中,采用了批处理和MapReduce工作流的方式,以误差和准确率来评估预测模型,解决了神经网络在处理海量数据时训练时间长等问题；实验表明,该方法可以在不影响准确度的前提下,大大缩短运行时间,提高预测效率。

Abstract:

Distributed storage and Distributed prediction method for flash flood forecasting disaster forecasting system of Rainfall data is researched. Focused on the rapid growth of the collected Hydrological data and the demands for prediction accuracy and timeliness of forecasts is increasing, respectively used Hadoop distributed file system to store data and use MapReduce framework and the genetic algorithm to optimize the number of hidden layer nodes and the weights as well as the thresholds of the network to predict data. Based on multi-factor flash flood disaster rainfall BP neural network prediction model, combining the characteristics of genetic algorithm can achieve global optimization to optimize the number of hidden layer nodes and the weights as well as the thresholds of the network,and in the procedure of data parallel processing adopted the way of batch mode and MapReduce workflow, and used the error and the accuracy to evaluate the prediction model,which solve the problem of network training time when the neural network in dealing with mass data. Experiments show that this method can greatly reduce the running time without affecting the accuracy, and improve prediction efficiency.