Virtual colonoscopy accuracy suffers when radiologists work too fast

There is much talk, and some evidence, that innovations such as CAD and primary 3-D reading will soon smooth out the rugged learning curve for virtual colonoscopy providers. But no matter what legions of computer programmers might come up with, at the end of the day it’s still radiologists who are responsible for the data -- and a new study suggests that experience and confidence can work against them.

At the virtual colonoscopy sessions of last week's RSNA meeting in Chicago, Dr. Martina Morrin from the Beth Israel Medical Center in Boston presented a study that evaluated the impact of experience on the correct identification of colorectal masses and polyps. In some cases, experienced radiologists who read too fast found their sensitivity had decreased.

"The limitations of CT colonography include high interobserver variability, difficulty in detecting flat polyps, inadequate bowel distension, and poorly prepared colons," she said. The group’s retrospective study was designed to look at interobserver variability from the viewpoint of reader experience.

Morrin, along with Dr. Jacob Sosna, identified 30 VC exams that had full endoscopic correlation with conventional colonoscopy. The cases included 3 masses and 20 polyps (7 10-19 mm, 8 from 5-9 mm, 4 < 5 mm) as well as 14 normal exams. Some of the cases were technically inadequate, with problems such as poorly distended segments.

The exams were then provided to six additional board-certified gastrointestinal radiologists for independent interpretation, two each with experience of no prior VC reading, two who had read 20-30 cases, and two who had seen more than 50-60 exams. Each reader listened to a lecture on VC interpretation and attended a workstation training session before beginning. The cases were interpreted primarily by using 2-D axial slices and 2-D multiplanar reformations, with 3-D endoluminal surface-shaded images used selectively, Morrin said. Interpretation time was recorded for each study, and the results were analyzed both by polyp and by patient.

Overall per-lesion results showed sensitivity of 95%, 59%, 44%, and 33% for masses ±20 mm and larger (n=3), large polyps 10-19 mm (n=7), medium polyps 6-9 mm (n=8), and polyps 5 mm and smaller (n=4), respectively. Overall per-patient specificity for the same lesion sizes was 95%, 91%, 77%, and 74%.

Overall, the most experienced radiologists had the best specificity, while those with moderate experience had the best sensitivity for polyp detection. And the most experienced readers took the least time to read the studies (averaging 5-9 minutes per study).

"The best results were from readers who took over 20 minutes to read the study," regardless of their level of experience, she said, adding that "reading CTC in less than 10 minutes significantly impairs performance among experienced readers."

Morrin cautioned that the pilot study relied on very small numbers of polyps, patients, and readers, and said the group would follow up with a larger survey. Still, she said, the results suggest that diagnostic accuracy is probably more closely related to reading time than reader experience. In response to a question from the audience, she offered a personal perspective on the impact of experience with virtual colonoscopy studies.

"I know myself when I started reading these studies several years ago I took a long time, and my sensitivity was quite high in the beginning," she said. "And then I got more confident and I thought I didn’t need as much time, and my sensitivity dropped off during that period of time. And then when I got the follow-up confirmations on misses, for example, my sensitivity went back up again, and now it has plateaued."

By Eric Barnes
AuntMinnie.com staff writer
December 12, 2003

Copyright © 2003 AuntMinnie.com