A Multi-Criteria Method for Selecting the Right Automated Software Testing Tool

Abstract

Software testing plays a vital role in the Software Development Life Cycle (SDLC) as it validates and verifies the software being built. To improve efficiency, automation testing has become widely adopted. In the world of software testing, selecting test automation tools can be daunting. Even for professionals, considerable time is often spent identifying and evaluating suitable tools before testing can proceed. However, its success depends heavily on selecting appropriate tools. With countless options available, this selection process often prolongs the testing phase. This study addresses the challenge by identifying key criteria for tool selection and developing a decision-making framework based on the Multi-Criteria Decision Making (MCDM) approach, specifically the Weighted Sum Model (WSM). The framework was implemented as an application that evaluates and ranks potential tools by assigning weights to defined criteria. The framework was tested on 14 widely used automation tools, including industry-recognized options such as Selenium, Appium, and Cypress, across seven evaluation criteria: ease of use, programming skills, scalability and performance, flexibility and customization, cost and licensing, vendor stability and product support, and reviews and recommendations. To assess effectiveness, feedback was collected from 52 respondents. Findings showed that 67% agreed the framework is useful for the software industry, and 81% considered the selected criteria highly relevant. Additionally, 62% confirmed the framework produced accurate results, while 56% affirmed its efficiency in supporting tool selection decisions. These results demonstrate that embedding an MCDM method in tool selection provides a structured and evidence-based approach. The developed framework not only streamlines decision-making but also contributes to faster and more reliable adoption of automation tools, ultimately improving the overall efficiency of software testing within the SDLC.