基于大数据应用的地质灾害数据存储策略

Geological Disaster Data Storage Strategy based on Big Data Application

摘要:

针对人工智能算法和大数据技术在地质灾害监测和预警上的应用需求,基于分布式文件系统(HDFS)和列式存储非关系型数据库(HBase)提出了地质灾害相关数据的存储策略。分析了地质灾害监控系统、地质灾害预测预报系统所需使用数据的数据种类、数据格式、数据容量、数据频率及数据增长速度等信息。从数据粒度大小的角度来对数据进行分类和组织,对不同粒度的数据设计了不同的存储模式,以实现高效的存取效率。根据数据的应用特性对数据进行类别划分,为不同类型的数据提供不同的存储结构和访问接口,以获得最优的数据访问性能。

Abstract:

For the application of artificial intelligence and big data technology in geological hazard monitoring and forecast, a geological hazard data storage strategy is built based on distributed file system (HDFS) and column storage non-relational database (HBase). The data type, data format, data capacity, data frequency and data growth rate of geological hazard monitoring system and geological hazard prediction system are analyzed. Data is classified and organized from the perspective of data granularity, and different storage modes are designed for different granularity data to achieve efficient access efficiency. Data is classified according to the application characteristics of data, and different storage structures and access interfaces are provided for different types of data to obtain optimal data access performance.