The effect of Bellwether analysis on software vulnerability severity prediction models

Abstract

Vulnerability severity prediction (VSP) models provide useful insight for vulnerability prioritization and software maintenance. Previous studies have proposed a variety of machine learning algorithms as an important paradigm for VSP. However, to the best of our knowledge, there are no other existing research studies focusing on investigating how a subset of features can be used to improve VSP. To address this deficiency, this paper presents a general framework for VSP using the Bellwether analysis (i.e., exemplary data). First, we apply the natural language processing techniques to the textual descriptions of software vulnerability. Next, we developed an algorithm termed Bellvul to identify and select an exemplary subset of data (referred to as Bellwether) to be considered as the training set to yield improved prediction accuracy against the growing portfolio, within-project cases, and the k-fold cross-validation subset. Finally, we assessed the performance of four machine learning algorithms, namely, deep neural network, logistic regression, k-nearest neighbor, and random forest using the sampled instances. The prediction results of the suggested models and the benchmark techniques were assessed based on the standard classification evaluation metrics such as precision, recall, and F-measure. The experimental result shows that the Bellwether approach achieves F-measure ranging from 14.3% to 97.8%, which is an improvement over the benchmark techniques. In conclusion, the proposed approach is a promising research direction for assisting software engineers when seeking to predict instances of vulnerability records that demand much attention prior to software release.