The Significant Effect of Parameter Tuning on Software Vulnerability Prediction Models

Abstract

Vulnerability prediction is one of the critical issues for researchers in the software industry. Technically, a vulnerability predictor is a machine learning model trained to identify vulnerable and non-vulnerable modules. Recent studies have shown that the performance of these models can be affected when the default parameter settings are used. Unfortunately, most studies in literature present their results using the default parameter settings. This study investigates the extent to which parameter optimization affect the performance of vulnerability prediction models. To evaluate our procedure, we conducted an empirical study on three open-source vulnerability datasets, namely Drupal, Moodle and PHPMyAdmin using five machine learning algorithms. Surprisingly, we found that in all cases of the 3 datasets studied, our models provided a significant increase in precision and accuracy against the benchmark study. In conclusion, software engineers can use the results obtained from this study when building data miners for identifying vulnerable modules.