Interobserver variability generally low at lung screening follow-up

Jan 27, 2009

Radiologists who perform CT follow-up screening of smokers generally agree on whether a nodule has grown at follow-up and whether the case should be considered positive or negative, according to a study of subjects screened twice in the National Lung Screening Trial (NLST). But there was less agreement on what should be done with the results.

CT's ability to identify lung nodules as small as 1 mm in diameter and assess nodule growth over time lies at the heart of hopes that CT screening of smokers and former smokers can identify early-stage lung cancers in time to effect a curative resection.

While routine CT screening is not yet recommended by major healthcare organizations, myriad studies have examined the modality's screening potential among long-term smokers. In the case of the multicenter randomized NLST trial, CT is compared to chest radiography for its potential to detect early-stage cancers.

There are well-established guidelines for nodule follow-up, but evaluating nodules remains an inherently subjective task. The NLST protocol involves annual CT or x-ray screening of smokers with more than 30 pack-years of exposure, with more aggressive follow-up recommended in cases where the nodule has grown during the year since the first scan was acquired.

At the 2008 RSNA meeting, Dr. Satinder Singh from the University of Alabama at Birmingham discussed his group's study of 100 individuals who underwent CT screening in the NLST trial.

"The task of the radiologist who is integrating these serial studies, especially on second CT scan is, one, to detect any new nodules, and then to evaluate the status of any previously known nodule and determine whether it has changed in size or attenuation or changed in contour," Singh said. "Based on these evaluations, they recommend what needs to be done," such as continued screening at one-year intervals, a PET exam, or biopsy of the nodule.

Because the extent to which reader variability affects these decisions is poorly understood, the study examined changes in pre-existing nodules from baseline (T0) to second (T1) screening study among the NLST participants who had CT screening, "and also to determine inter-reader variability in follow-up or diagnostic recommendations based on the recommendations made," Singh said.

The team reviewed 100 examinations in which the original interpretation found a noncalcified nodule 4 mm or larger at the T1 CT screening in the same location as at T0. Image sets of the100 cases containing the nodule from both datasets were interpreted on a two-monitor PACS workstation by nine NLST radiologists.

"Only a limited number of images were provided for interpretation from both CTs to minimize distraction, and the nodule to be identified from the T1 [second screening] was identified by a marker," he said. "Readers were asked to provide a bidimensional diameter measurement of the nodule; any change or absence of any change in size, attenuation, or contour; tell us what is the confidence level of the observation; and finally, recommendations for follow-up."

The results showed that agreement on measurements of nodules was excellent, with intraclass coefficients of 0.91 to 0.96 for T0 and T1 diameters. Agreement on the yes-or-no question of whether the nodule had grown was also very good, with multirater kappa (MK) for discrete variables of 0.75 (range, 0.69-0.79).

However, there was less agreement with regard to changes in attenuation (MK = 0.33) and nodule margins (MK = 0.36). Moreover, agreement on overall screening results (suspicious for lung cancer, suspicious but no significant change, not suspicious) was only moderate, with MK of 0.48 (0.45-0.51).

Except for a single outlier, agreement between screening results and diagnostic recommendations was excellent at 98% (84%-100%). However, the diagnostic recommendations (none, low-level, and high-level procedures) had only fair agreement between readers, with MK of 0.24 (0.22-0.26).

"To further evaluate why this recommendation was so low, we tried to compare CT results with follow-up recommendations on the charts," Singh said. "People who said the CT study was negative on T1 recommended mostly continued screening. In patients labeled positive or no change, almost 20% were recommended for short-term follow-up."

But in patients labeled both positive with changes at follow-up, almost 12% were not referred for aggressive follow-up, Singh said.

"There was good and moderate agreement, especially for nodule growth and interpretation among the readers," he concluded. "There was less satisfactory agreement for diagnostic recommendations, and this is probably is due to the difficulty in assessing the changes in attenuation in these small nodules."

The variability among readers is consistent with previous studies measuring inter-reader agreement. The over-representation of nodules with changes limits the extent to which these results can be extrapolated to the CT arm of NLST as a whole, the authors wrote in an abstract.

Following additional research, the researchers plan to follow-up with specific recommendations for the various findings at follow-up, Singh said.

By Eric Barnes
AuntMinnie.com staff writer
January 28, 2009

New CT method measures airway calcium to predict lung cancer, December 16, 2008

PET SUV measurements vary between facilities, December 10, 2008

Lung CAD holds up well in low-dose MDCT studies, November 25, 2008

Measuring lung nodule growth beyond variability, November 5, 2007