面向静态分析的软件测试工具评估方法研究

Study on Evaluation Method of Software Testing Tools for Static Analysis

摘要:

软件是否可靠运行将直接影响系统的可靠运行,急需有手段保障软件质量。静态分析因具有全自动运行、能更早实施、不需要执行程序等特点,在软件测试领域得到了广泛的使用,已成为保障软件质量的重要手段。测试工具可以极大地提高软件测试的效率。目前面向静态分析的软件测试工具数量众多,不同的工具具有不同的特点和缺陷检测能力,且都存在不低的误报率和漏报率,如何评估和选择测试工具成为软件静态分析时亟待解决的问题。基于CWE缺陷类型的基准测试集和精确度、召回率、F1-Score、CWE覆盖率以及Overall-Score的评估指标,详细阐述了面向静态分析的软件测试工具评估流程,结合CppCheck、TscanCode和Flawfinder三个开源软件静态分析工具开展了案例应用,为软件静态分析工具的评估与选择提供指导和参考。

Abstract:

Whether the software runs reliably will directly affect the reliable operation of the system, and there is an urgent need to have means to guarantee software quality. Static analysis has been widely used in the field of software testing due to its features such as fully automatic operation, earlier implementation, no need to execute code, etc. It has become an important means to guarantee software quality. Testing tools can greatly improve the efficiency of software testing. At present, there are many software testing tools for static analysis, different tools have different characteristics and defect detection capabilities, and all of them have not low false positive rate and false negative rate, how to evaluate and select the testing tool has become an urgent problem to be solved. Based on the benchmark test set of CWE, and the five evaluation metrics of precision, recall, F1-Score, CWE coverage and Overall-Score, the evaluation process of software testing tools for static analysis is elaborated. A case study was conducted using three open-source software static analysis tools, CppCheck, TscanCode, and Flawfinder, which provides guidance and reference for the evaluation and selection of software static analysis tools.