试飞数据平台数据架构设计与应用

Design and application of Flight test data processing and analysis platform Architecture

摘要:

试飞数据是民机飞行试验的重要产物,具有测量参数数量大,数据体量大,飞行试验数据与试飞任务信息关联性强等特征,支撑飞机型号取证与设计优化等任务。试飞数据平台数据架构对多源异构数据集成接入、多类形态数据存储管理、多种层次数据处理分析等技术进行了研究,采用湖仓一体的关键技术和方法打造试飞数据全集。试飞数据接入采用流批一体的数据处理技术,融合Spark和Flink主流数据处理引擎,具备试飞数据快速入库能力。提出按秒聚合方法,具备PB级多维度试飞数据压缩存储功能,存储性能提升近10倍；采用以秒为索引条件支持快速检索,强化数据湖查询能力。研究数据仓库技术,设计试飞数据多层数据模型,具备多维信息精细查询,多层数据灵活钻取,多功能自定义函数集成等功能,并成功应用在某型国产民机的飞行试验数据管理中,服务于试飞数据用户,提高了试飞数据管理效率与试飞数据应用价值。

Abstract:

Flight test data is an important output of civil aircraft flight tests, characterized by a large number of measurement parameters, large data volume, and strong correlation between flight test data and test flight mission information, supporting aircraft type certification and design optimization tasks. The data architecture of the flight test data platform integrates multiple heterogeneous data sources, manages various types of data in different formats, and provides multi-level data processing and analysis functions, creating a unified data lakehouse for flight test data. The data processing technology used for flight test data integration adopts a hybrid approach of stream-batch integration, incorporating mainstream data processing engines such as Spark and Flink, with the ability to quickly ingest flight test data into the data platform. The platform proposes a method of aggregation at the second level, with PB-level multidimensional flight test data compression and storage capabilities, supporting fast retrieval based on second-level indexing conditions and enhancing data lake query capabilities. Data warehousing technology is also studied, with the design of a multi-layer data model for flight test data, supporting fine-grained queries for multidimensional information, flexible drilling down into multiple layers of data, and integration of custom functions, which has been successfully applied in the management of flight test data for a certain type of domestic civil aircraft, serving flight test data users and improving the efficiency and value of flight test data management.