Radiologists detect more polyps when computer-aided-detection (CAD) software is required to mark lesions on both prone and supine views at virtual colonoscopy (also known as CT colonography or CTC) before findings can be deemed positive.
The research could eventually lead to greater clinical utility for CAD, and potentially less fatigue for readers, who would be burdened with interpreting fewer CAD detections in each patient, according to the study team from the U.S. National Institutes of Health (NIH) in Bethesda, MD, and several other centers.
Lead author on the paper was Ron Summers, MD, PhD, along with colleagues from the NIH; the National Naval Medical Center, also in Bethesda; the Walter Reed Army Medical Center in Washington, DC; and the University of Wisconsin School of Medicine and Public Health in Madison. The study was published online before print on June 12, 2010, in Academic Radiology.
Independent analysis
Current CAD systems analyze supine and prone CTC scans independently. The reader then analyzes the findings after considering the size, location, and morphology of the CAD marks, considering a polyp "detected" if it is marked by CAD on either or both views, the authors explained.
CAD detection on both supine and prone VC views hasn't been a focus of either researchers or CAD developers, although conspicuous polyps are usually detected on both views. Polyps might be missed on one or both scans for several reasons, including image noise, poor distention, poorly tagged fluid or stool, or changes in polyp shape, such as a flatter appearance when the patient is turned over for the second scan, the authors noted.
The study focused on the more challenging 6- to 9-mm polyps for which CAD has been shown to provide a benefit in observer studies, Summers and colleagues wrote. The data included 33 subjects (20 men and 13 women, ages 47 to 76, mean 59.4 years) selected from previous CTC and CAD trials of consecutive screening patients.
Patients underwent cathartic bowel prep that included a sodium phosphate preparation, bisacodyl, barium (2.1%), and diatrizoate meglumine and diatrizoate sodium (Bracco Diagnostics, Wayne, NJ) in divided doses.
CT images were acquired in the prone view using a four- or 16-detector-row CT scanner (GE Healthcare, Chalfont St. Giles, U.K.). Scanning parameters included 1.25- or 2.5-mm collimation, 1-mm reconstruction intervals, 100 mAs, and 120 kVp.
Interpretation was performed using the group's previously validated CAD software in first-reader mode (i.e., the radiologists reviewed only the CAD marks). Four experienced radiologists read the 33 CTC cases, 21 of which had a single adenoma 6 mm to 9 mm in size. None of the positive cases had more than a single true-positive lesion.
Patients also underwent same-day optical colonoscopy using a segmental unblinding technique to create an enhanced reference standard using both colonoscopy and CTC results.
Each polyp visualized on optical colonoscopy was also located on the prone and supine CTC images using V3D Colon software (Viatronix, Stony Brook, NY). To be considered a match between both techniques, the polyp had to be located within the same or an adjacent colonic segment and measure within 50% of the size.
The radiologists viewed each case twice, with different sets of CAD marks applied for each of the two readings. In the first reading, a true-positive CAD mark for the same polyp was displayed on both the supine and prone scans (a double-mark reading). In the second reading, a true-positive CAD mark was displayed either on the supine or prone scan, but not both (a single-mark reading). The true-positive marks were randomized between readings, and there was at least a one-month delay between readings to minimize recall bias.
The researchers determined sensitivity and specificity, and performed receiver operating characteristic (ROC) and multiple-reader multiple-case analyses.
Higher sensitivity for double CAD
The results showed that per-patient sensitivities were significantly greater in the double-mark readings compared to the single-mark readings for three of four radiologists. Specificities were also greater for two of the four radiologists in double-mark reading, but the differences for both individual readers and the average reader were not statistically significant, Summers and colleagues wrote.
Average per-polyp sensitivities were 60% (38% to 81%) for single-mark reading versus 71% (52% to 91%) (p = 0.03) for double-mark reading, respectively. The areas (95% confidence intervals) under the ROC curves (AUCs) were 0.76 (0.62 to 0.88) for single-mark and 0.79 (0.58 to 0.96) for double-mark reading, and were not statistically significant. Specificities were similar for the single-mark compared with the double-mark readings, the authors wrote.
"In this study, readers found medium-sized adenomas 11% more often on average when the polyp was marked by CAD on both the supine and prone scans rather than on only one scan," Summers and his team wrote. "This sensitivity increase is large and potentially clinically highly relevant, particularly if the findings translate to the concurrent or second-reader paradigms."
The variation in sensitivity was large among the readers, and the differences were statistically significantly different for the double-mark readings, they noted.
Several factors in the study design may have affected its (relatively low) sensitivities, including use of the CAD-first mode and the selection of difficult-to-detect medium-sized polyps, rather than both medium and large as one finds in screening studies, Summers and colleagues wrote.
In addition, the AUCs and specificities did not significantly change when double-mark CAD was used, a result the authors suggested might be due to particular study findings, including CAD marks on a hyperplastic polyp, an inverted diverticulum, and a polyplike finding, all considered false-positive by the criteria of this study.
"The lack of a significant change in specificity suggests that any potential downside to double-mark CAD presentation is small," the authors wrote.
Virtual colonoscopy is almost always performed using both prone and supine views because of the proven benefit in sensitivity and specificity. When both views are used, poorly distended segments can become well-distended, and residual fluid and stool can shift, rendering polyps visible, the group noted.
"However, the effect on radiologist readings of the use of CAD that detects the polyp on only one versus both scans has not previously been investigated," they wrote. "Different CT colonography CAD systems differ in their ability to detect polyps on both scans versus one or no scans."
These differences are not well-documented or emphasized in the literature, however, and the performance of CAD is usually determined by evaluating how many polyps are detected in either view. In the present study, CAD marks were withheld from the radiologists to simulate the distinction between detection in prone and supine views and evaluate the importance of CAD's marking polyps on both views.
"Our results indicate that this distinction matters for clinical interpretation of CTC images and that there is a benefit to having CAD cue the radiologist to a potential polyp's location on both scans," Summers and colleagues wrote.
Although the reasons behind improved performance by marking both views are unclear, the possibilities "include a perceptual benefit, because the radiologist has in essence two chances to identify the polyp, and a reduction in fatigue because the effort required to find the polyp on the alternate view is reduced," they wrote. "The marking of a potential abnormality on multiple images of the same body part may have reinforcing benefits through repetition that facilitates the radiologist's visual perception of the abnormality and improves diagnostic confidence."
The study's limitations included its small dataset, the use of a fixed number of CAD marks for each scan, and use of the first-reader CAD reading mode to minimize any effect of variations in the number of CAD marks.
"Double-mark display of CAD findings led to higher average per-polyp sensitivity without a statistically significant adverse effect on specificity for detecting 6- to 9-mm adenomas," the study authors concluded.
A CAD researcher comments
In an editorial accompanying the study, Janne Näppi, PhD, from Massachusetts General Hospital in Boston said that double matching may represent another tool for improving CAD performance, which until now has focused on reducing false-positive detections.
In practice, most CAD users don't match both sets of detections even though they're expected to, and in this study the use of "dual-matching" true-positive CAD prompts "yielded a significant improvement in the average detection sensitivity," Näppi wrote.
The study doesn't explain how this occurred, though the authors note that observing CAD marks on both views reduces perceptual error and reader fatigue while improving diagnostic confidence, Näppi wrote.
A key limitation is the study's "largely artificial" design, Näppi wrote. "Current [CAD] systems are not able to match patient scans or detect lesions on both scans reliably enough to provide the kind of 'dual-matching' [CAD] prompts that were simulated in the experiment, and the 'first-reader' technique of the study is not recommended for use with [CAD] because of its potentially detrimental effects on reader performance," he wrote.
The bottom line is that CAD systems are going to have to work harder if they want to gain the confidence of computer-assisted readers. This will require new efforts to develop CAD technologies for minimizing false positives, increasing sensitivity, and determining how best to display the information.
By Eric Barnes
AuntMinnie.com staff writer
July 14, 2010
Related Reading
VC CAD: Projection-based features reduce false positives, March 10, 2010
VC CAD application does well alone and as reader aid, February 15, 2010
New VC CAD algorithm reveals submucosal colon cancers, February 4, 2010
VC screening effective in Medicare-age screening cohort, January 27, 2010
VC CAD nabs undetected polyps in jumbo screening study, October 29, 2009
Copyright © 2010 AuntMinnie.com
