机场不正常事件实体检测与识别方法研究

Research on Detection and Recognition Method of Airport Abnormal Event Entities

摘要:

民航安全自愿报告系统收集的海量故障报告以非结构化文本形式存储,不便于相关人员针对大量不正常事件加以分析并采取控制措施；命名实体识别技术可以将海量非结构化文本中的关键要素进行检测和识别,抽取成类别分明的结构化信息,作为进一步分析不正常事件并加以控制的基础工作；将机场不正常事件报告作为研究对象,提出了一种基于神经网络的中文命名实体识别模型,对文本进行了结构化处理；针对随机选用的训练样本一些实体类别分布比较稀疏和人工标注费时费力的问题,提出了基于模型预测分数的样本选择策略,实现了预标注样本的高效筛选；经过实验验证,该模型与BiLSTM_CRF模型、BiLSTM_self-attention_CRF模型相比F1值均提高了约6个百分点,该样本选择策略明显提高了人工标注效率,筛选出足够多的含有稀疏实体的样本。

Abstract:

The massive reports of fault events collected by the civil aviation safety reporting system are stored in the form of unstructured texts, which are not convenient for the relevant personnel to analyze and take control measures for a large number of abnormal events. Named entity recognition technology can detect and identify the key elements in the massive unstructured texts and extract them into structured information with clear categories, which can be used as the foundation work for further analysis and control of abnormal events. As the Airport abnormal events reports are taken as the research object, a neural network-based Chinese named entity recognition model is proposed to structure the texts. For the problems of sparse distribution of some entity categories of randomly selected training samples and time-consuming and laborious manual labeling, a sample selection strategy based on model prediction scores is proposed to achieve efficient screening of pre-labeled samples. After experimental validation, the model improves the F1 value by about 6 percentage points compared with the BiLSTM_CRF model and BiLSTM_self-attention_CRF model, and this sample selection strategy significantly improves the manual annotation efficiency and screens out enough samples containing sparse entities.