READ-DVT is a validated artificial intelligence algorithm developed by Harvard researchers that automatically identifies deep vein thrombosis (DVT) from unstructured ultrasound reports with 93-99% accuracy across multiple medical centers, potentially saving clinicians significant time and improving diagnostic efficiency in emergency departments.
- READ-DVT achieved 93.4% sensitivity and 99.8% specificity on test sets, with external validation showing 98% sensitivity and 99.5% specificity after optimization
- The algorithm was developed and validated using 9,249 venous ultrasound reports from five academic medical centers including Harvard, Vanderbilt, and University of Texas Southwestern
- READ-DVT uses regular expression algorithm-based approach in structured query language to extract DVT information from free-text ultrasound reports
- The model outperforms traditional methods like diagnostic codes, anticoagulation prescriptions, and manual review which are often inaccurate or time-consuming
A multi-center validated AI model could help find deep vein thrombosis (DVT) in ultrasound reports, according to findings published August 7 in Thrombosis Research.
Researchers led by Drew Birrenkott, MD, from Harvard Medical School in Boston, MA reported success from their algorithm in identifying the presence or absence of DVT on unstructured imaging reports, which could save time for clinicians.
“It is a valid algorithm to define the DVT phenotype among [emergency department] patients undergoing venous ultrasound and may serve as a model for the general process of extracting information from unstructured [electronic health record] text,” Birrenkott and co-authors wrote.
Clinicians often need to manually review venous ultrasound reports for DVT analysis since the reports are often unstructured. Natural language processing meanwhile can be complex and difficult to place into the electronic health record (EHR) when identifying DVT presence from free text.
Birrenkott and colleagues developed and validated their regular expression algorithm-based approach in structured query language to identify DVT from unstructured ultrasound reports. They named their algorithm Regular Expression Aided Determination of Deep Venous Thrombosis (READ-DVT).
To develop READ-DVT, the researchers extracted 9,249 venous ultrasound reports from emergency departments at five academic medical centers: data from Vanderbilt University Medical Center in Nashville, TN; Massachusetts General Hospital in Boston; Barnes-Jewish Hospital in St. Louis, MO); the University of Texas Southwestern Medical Center in Dallas; and the University of Cincinnati Medical Center in Ohio. Of these reports, adjudicators labeled 1,398 (15.1%) to be positive for DVT. The team employed adjudicators to review reports for the presence or absence of DVT.
From there, the researchers split data from one medical center into test (n = 1,004 reports) and training sets (n = 512 reports) and performed external validation using data from the four remaining centers (n = 7,228 reports). They conducted multiple rounds of testing and validation including validation on new, unseen data when indicated.
READ-DVT achieved high marks in the test and external validation sets, though it underperformed on the team’s fourth external validation set.
Performance of READ-DVT on test, external validation sets | ||||||
Measure | Test set | Center 1 validation set | Center 2 validation set | Center 3 validation set | Center 4 validation set | Center 5 validation set |
Sensitivity | 93.4% | 93.7% | 100% | 97.2% | 83.8% | 98.9% |
Specificity | 99.8% | 99.2% | 99.5% | 98.9% | 99.4% | 99.4% |
F1 score | 0.96 | 0.93 | 0.99 | 0.96 | 0.90 | 0.98 |
“By using these data to iteratively update READ-DVT we found a simple addition to our algorithm vastly improved the model function with no change in performance in the original test, training, and initial three validation sets,” the team wrote.
From this, the team created a new external validation set from the same medical center as the fourth external validation set. The model achieved 98.0% sensitivity, 99.5% specificity, and an F1 score of 0.98.
The study authors highlighted that models like READ-DVT could help where techniques such as diagnostic codes, anticoagulation prescriptions, insurance claims data, and manual review “can be inaccurate or cumbersome.”
“The development of tools like READ-DVT is critical to applications such as clinical predictive modeling,” they wrote. “However, local validation of computer algorithms is necessary prior to use.”
Read the full results here.
