Improving strategic control of overconfidence and time spent on lengthy cases holds promise for reducing diagnostic errors and increasing workflow efficiency in radiology, according to analysis published May 26 in Radiology.
The study suggests there may be benefits in creating a system for residents to opt out of challenging cases earlier, essentially streamlining consultations with experienced radiologists, according to Eyal Bercovich, MD, and Rakefet Ackerman, PhD, from the Herta and Paul Amir School of Medicine at the University of Haifa in Israel.
Bercovich and Ackerman used metacognitive frameworks to call attention to the prevalence of overconfidence and time wastage in high-stakes clinical settings. They analyzed how the "hard-easy effect" phenomenon and temporal thinking dynamics manifest in radiologic decision making for experienced radiologists and nonradiologists.
"Recent frameworks emphasize effective opting out -- training students to recognize when to stop and seek help -- as a vital efficiency skill," noted Bercovich and Ackerman. "Our study provides empirical evidence to support these calls, potentially encouraging interventions that create opting-out equivalents by streamlining consultation and referral processes."
Although some cognitive biases have been acknowledged, such as "satisfaction of search," biases stemming from the metacognitive processes have been mostly overlooked, the authors explained. To bridge the gap, the researchers applied metacognitive theory to clinical procedures (2,806 decisions) to compare decision-making between radiologists proficient in bone radiographs (n = 28) and nonradiologist medical professionals (n = 50).
The stimulus set consisted of three practice radiographs to orient participants to the task, followed by 48 hospital-sourced radiographs chosen to span a range of diagnostic difficulty (20% to 100% success rates). They included 42 images of the limbs (forearm, wrist, and knee) alongside six of the core body (ribs and pelvis).
Ground truth for bone lesion presence or absence was established through follow-up advanced imaging (CT or MRI) and confirmed by two board-certified musculoskeletal radiologists (each with over eight years of experience).
The results expose metacognitive commonalities and differences between expertise levels, Bercovich and Ackerman said. In both groups, quicker diagnoses were more successful and accompanied by better-calibrated confidence than were slower responses.
In terms of the hard-easy effect, both radiologists and nonradiologists showed well-calibrated confidence for easy cases (success, approximately 98%; confidence, approximately 93%) but overconfidence (>40%) for difficult cases (success, approximately 37%; confidence, approximately 80%), according to the results.
Regardless of response time (RT), radiologists were more successful and better calibrated than nonradiologists. Overall, radiologists demonstrated higher diagnostic success (73.59% versus 64.71%; p < 0.001), less overconfidence (11.52% versus 16.52%; p = 0.03), and better resolution (γ = 0.37 versus 0.23; p = 0.04).
Radiologists were notably less overconfident and exhibited significantly better resolution than nonradiologists, the authors explained.
Bar graphs show mean success (blue) and confidence (orange) according to diagnostic response (lesion present or absent, shown as “present” and “absent,” respectively) and next-step decision (submit, additional tests, and consult a senior) for each group (radiologists and nonradiologists). Error bars indicate standard errors of the mean. Lesion-absent diagnoses were more prone to error and overconfidence than were lesion-present diagnoses.Caption and image courtesy of RSNA.
The most worrisome were the cases of missed lesions by nonradiologists who asked for additional tests but did not lower their confidence in these diagnoses, Bercovich and Ackerman wrote. Lesion-absent diagnoses were more prone to error and overconfidence than were lesion-present diagnoses, they added.
"Metacognitive pitfalls undermined the diagnostic process but were attenuated among radiology experts, who showed improved success, self-monitoring, and time regulation -- expertise facets rarely documented in medical contexts," Bercovich and Ackerman explained.
However, findings were based only on bone radiographs. Therefore, the researchers recommended examining whether these metacognitive dynamics generalize to other body parts and other modalities. They also envisioned PACS nudging users when viewing time lengthens, prompting a suggestion to consider a second opinion.
Furthermore, "lengthy thinking during training is desirable for improving professional skills," the authors concluded. "Residency training could incorporate time-aware case debriefs combined with guidance toward early consultation pathways for cases accompanied by pronounced uncertainty."
Find the complete paper here.
