An artificial intelligence (AI) algorithm could be utilized in low-resource settings to stratify a patient's risk of having COVID-19 based on their chest x-rays, according to research published online January 13 in Intelligence-Based Medicine.
After assessing a modified version of a commercially available deep-learning algorithm developed initially to detect tuberculosis, researchers led by Diego Hipolito Canario of the University of North Carolina at Chapel Hill found in a clinical study that the software yielded a high level of accuracy in detecting radiographic abnormalities indicative of COVID-19 on chest x-rays.
What's more, the algorithm's risk scores had high positive predictive value and negative predictive value in comparison with real-time reverse transcription polymerase chain reaction (RT-PCR) testing in those patients who received the tests.
"The [AI] algorithm has the potential to provide benefit in guiding medical management of patients suspected of having COVID-19 who present with a high likelihood of disease, and where timely viral testing is not feasible due to limited resources," the authors wrote.
In an attempt to assess the performance of AI to provide medical triage of possible COVID-19 patients in resource-limited settings, the researchers evaluated a modified version of the qXR deep-learning algorithm (Qure.ai) after its deployment at a private hospital in Mexico City, Mexico during the initial stages of the COVID-19 pandemic in April to May 2020.
After receiving additional training on COVID-19 patients, the modified algorithm -- called M-qXR -- produces a risk score that stratifies patients into no-risk, low-risk, medium-risk, or high-risk categories for COVID-19 based on the presence of pulmonary opacities and consolidations with a bilateral and peripheral distribution, according to the researchers. The institution used the software to help stratify at-risk patients to receive further testing.
Examples of chest radiographs evaluated by the M-qXR algorithm. M-qXR assigned a high (a), medium (b), and low (c) COVID-19 risk score based on the presence of radiographic abnormalities on the chest radiographs presented. Images and caption courtesy of D. Hipolito Canario et al and Intelligence-Based Medicine through Creative Commons Attribution 4.0 International License.In the first part of the study, the researchers compared the performance of the algorithm with a majority consensus interpretation by four radiologists with an average of 22 years of experience. Each of the radiologists independently read the 625 chest x-rays over the eight-week study period.
Overall, the algorithm agreed with 98% of the radiologist interpretations and yielded strong results for identifying pulmonary opacities and the presence or absence of pulmonary consolidation.
| Performance of M-qXR for identifying chest x-ray findings indicative of COVID-19 | ||
| Pulmonary opacities | Presence or absence of pulmonary consolidation | |
| Accuracy | 94% | 88% |
| Sensitivity | 94% | 91% |
| Specificity | 95% | 84% |
| Positive predictive value | 99% | 89% |
| Negative predictive value | 88% | 86% |
Next, the researchers compared the AI model's results with 113 patients confirmed from a PCR test to have COVID-19. After creating a co-occurrence matrix between the AI algorithm's COVID-19 risk score and COVID-19 PCR test results in 113 PCR-confirmed COVID-19 cases, the researchers also found that a medium to high-risk score yielded a positive predictive value of a positive COVID-19 PCR test result in 89.7% of cases and a negative predictive value of 80.4%.
"We also believe that M-qXR's ability to localize and quantify the affected regions on CXR, will enable us to monitor for progression of infection and evaluate response to treatment in future studies," the authors wrote.
The researchers said their research could also help further validate future studies utilizing a similar risk-stratification approach for a triage setting.
