Patient-level performance evaluation of a smartphone-based malaria diagnostic application
Loading...
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
Malaria Journal
Abstract
Background Microscopic examination is commonly used for malaria diagnosis in the field. However, the lack of
well-trained microscopists in malaria-endemic areas impacted the most by the disease is a severe problem. Besides,
the examination process is time-consuming and prone to human error. Automated diagnostic systems based on
machine learning ofer great potential to overcome these problems. This study aims to evaluate Malaria Screener, a
smartphone-based application for malaria diagnosis.
Methods A total of 190 patients were recruited at two sites in rural areas near Khartoum, Sudan. The Malaria Screener
mobile application was deployed to screen Giemsa-stained blood smears. Both expert microscopy and nested
PCR were performed to use as reference standards. First, Malaria Screener was evaluated using the two reference
standards. Then, during post-study experiments, the evaluation was repeated for a newly developed algorithm,
PlasmodiumVF-Net.
Results Malaria Screener reached 74.1% (95% CI 63.5–83.0) accuracy in detecting Plasmodium falciparum malaria
using expert microscopy as the reference after a threshold calibration. It reached 71.8% (95% CI 61.0–81.0) accuracy
when compared with PCR. The achieved accuracies meet the WHO Level 3 requirement for parasite detection. The
processing time for each smear varies from 5 to 15 min, depending on the concentration of white blood cells (WBCs).
In the post-study experiment, Malaria Screener reached 91.8% (95% CI 83.8–96.6) accuracy when patient-level results
were calculated with a different method. This accuracy meets the WHO Level 1 requirement for parasite detection.
In addition, PlasmodiumVF-Net, a newly developed algorithm, reached 83.1% (95% CI 77.0–88.1) accuracy when
compared with expert microscopy and 81.0% (95% CI 74.6–86.3) accuracy when compared with PCR, reaching the
WHO Level 2 requirement for detecting both Plasmodium falciparum and Plasmodium vivax malaria, without using the
testing sites data for training or calibration. Results reported for both Malaria Screener and PlasmodiumVF-Net used
thick smears for diagnosis. In this paper, both systems were not assessed in species identifcation and parasite counting, which are still under development.
Description
Research Article
Keywords
Malaria microscopy, Computer-aided diagnosis, Automated screening, Machine learning