Email Print
Errors in reports are a 'sleeping giant of a problem'
By Rebekah Moan, AuntMinnie.com staff writer
November 28, 2012

CHICAGO - Errors due to voice recognition software are a "sleeping giant of a problem," a radiologist said during a question and answer session at RSNA on Tuesday afternoon. In any one line of his dictation, there's a 52% chance there will be an error, he added.

Radiologists used to use transcriptionists to generate and edit reports, but due to cost, transcriptionists have fallen by the wayside and voice recognition software used alone has become standard. However, voice recognition software inserts errors into the text that could have dire consequences.

"My reports -- and I try to be careful -- average seven errors per report, which go from punctuation to ludicrous," said Dr. Michael McNamara Jr. from Case Western Reserve University School of Medicine. "[Voice recognition software] inserts a 'no,' it drops a 'no' -- it's a very dangerous weapon and we have to use it very, very carefully," he said.

McNamara raised his concerns during a presentation by Dr. Nima Momenin from Wayne State University on the quality of voice recognition software versus using a transcriptionist for radiology reporting over time.

The study involved 100 chest x-ray reports from five radiologists sampled retrospectively for four categories:

  1. Traditional dictation and transcription method before voice recognition
  2. Voice recognition with transcriptionist correction
  3. Voice recognition with self-correction three months after implementation
  4. Voice recognition two years after the implementation of the continuous speech recognition system

The error rate using voice recognition with correctionists was slightly better than the pre-voice recognition rate, according to the researchers; however when the correctionist was removed, the error rate rose to 2.5 times higher in reports generated by voice recognition with self-correction.

The rate of errors after two years of using voice recognition rose to 1.5 times higher than three months after implementation of voice recognition. Fortunately, only two reports in the third category and one report in the fourth category contained major errors, Momenin said.

"Using voice recognition technology results in a high frequency of transcription errors compared to traditional transcription method, jeopardizing the quality of radiology reports," he added. "Most of the errors did not affect the core meaning of the report and are unlikely to alter patient management; however many made strangely worded reports."

Comparing reports made after two years using voice recognition with those generated three months after its implementation showed significant increase of the error rates, which brings into question the value of the "continuous learning" ability of the software, Momenin said.

"Recognition of deterioration in performance suggests that voice recognition must be carefully monitored after installation to maintain report quality, and that prolonged familiarity with the device is not sufficient to sustain report quality," he concluded.

New medical transcriptionists may adapt better to SR system editing, November 16, 2012

Study: Speech recognition produces errors in breast reports, September 21, 2011

Factors affecting error rate in speech recognition dictation, November 5, 2009

Radiologists underestimate overlooked SR report errors, January 23, 2009

Study: speech recognition boosts error rates in radiology reports, September 25, 2008


Copyright © 2012 AuntMinnie.com

Last Updated np 12/5/2012 11:46:18 AM

20 comments so far ...
11/28/2012 7:42:37 PM
dr cm nola
The software my group uses does not learn; it is as stupid as ever. Of course, all software is bought caveat emptor! Error mitigation is a real challenge with computer voice recognition. I work hard at editing the reports, but errors still appear in my final, signed reports. Our group's software is from a well known vendor. I prefer to call it "Talk Trash."

11/28/2012 7:57:29 PM
vonbraun
Seems like the software engineers would be able to program some logic into the VR output so that it would catch absurd phrases the software now spews out.

11/28/2012 8:10:49 PM
billainsworth
True, live transcriptionists make errors. But I'll be darned if I am going to play typist for the hospital and type my own reports. I will retire and work at Costco if they try to institute VR.

11/29/2012 5:59:38 AM
Raddocmed
Good luck with that. The hospital may well come to you and say either go with VR or take a hike.

11/29/2012 9:44:15 AM
MISTRAD
Yeah, good luck...VR is coming.  It is painful, no question.  I almost wish I could hire my own transcriptionist. 

Email Print