How to compare a noninvasive test to a gold standard

2001 06 06 15 10 10 706

Noninvasive tests are challenging radiology's gold standards across the board. In the rush to provide safer and cheaper alternatives to invasive procedures, colonoscopy is yielding to VC, ECRP becomes MRCP, and V/Q is sent looking for PE and DVT. But are the new tests always the best choice? Or are they expensive and ill advised, spared only by the steely determination of their proponents?

Dr. Richard Tello can calculate an answer. The associate professor of radiology, epidemiology, and biostatistics from the Boston University School of Medicine has developed simple yet plausible formulas that can weigh factors such as disease probability and complication rate, then apply the results to some of the gnarliest patient-management dilemmas.

The methods are designed to determine if a noninvasive test is cost-effective for a particular patient, or whether alternative management is more appropriate. In contrast, traditional decision analysis is far more complex, requiring sophisticated multilevel decision trees to account for the likelihood of various outcomes. The problem is that traditional formulas can be unwieldy, and often require information that's difficult to obtain in a clinical setting.

Speaking at the 2001 American Roentgen Ray Society meeting in Seattle, Tello aimed his group's simplified methodology at two sets of tests: virtual colonoscopy (VC) vs. colonoscopy, and magnetic resonance cholangiopancreatography (MRCP) vs. endoscopic retrograde cholangiopancreatography (ECRP).

"In this dilemma, the variables consist of the cost of the invasive gold standard (Co), the cost of the noninvasive test (Ct), the cost of missing the diagnosis (DX Cd), the cost of the complication by performing the gold standard (Cg), and of course, the rate of complications from performing the gold standard (R)," Tello said.

For the study, the researchers encoded the variables for the two sets of tests in Mathematica 2.0 software (Wolfram Research, Champaign, IL), varying the values over a diagnostic range in order to measure their impact. Decision trees were created to display the patient-management scenarios, and the process was applied to the two sets of tests, Tello said.

"The decision tree is: you've got a patient who's got to have a study done," he said. You can do nothing, in which case you just have to deal with the cost "P" -- and the cost is going to be the cost of misdiagnosis times one minus P for those who have the disease. The other option is to go directly [to] the gold standard test, in which case the cost will be the cost of the gold standard, plus the complication rate in those patients who are going to be exposed to it. Or you could do the noninvasive test, and if the test is negative you can then deal with the cost of misdiagnosis in false-negative cases, or you can go on to do the gold standard test."

2001 06 06 15 10 10 706

The cost-effectiveness model requires values for the sensitivity, specificity, and cost of each test, and optimally the probability of disease. Graphics courtesy of Dr. Richard Tello.

2001 06 06 15 06 36 706

In the first scenario, VC vs. colonoscopy, Tello used a colonoscopy complication rate of 7.8%, a "conservative" complication cost of $1,000, sensitivity and specificity of 1.0 for VC, and a pretest disease probability of 0.01% to calculate the cost-effective price of the noninvasive procedure.

"As long as you can pursue a VC at less than $1,067, it's basically a cost-effective option," Tello said. "If we increase the pretest probability to 0.03%, then the cost of colonoscopy has to drop down to less than $755."

In the second scenario, MRCP vs. ECRP, Tello gave MRCP a sensitivity and specificity of 1.0 and 0.5, respectively, and a pretest disease probability of 0.5. The gold standard had an 8% complication rate at a cost of $1,000 per complication, and a $4,000 cost of misdiagnosis. In this scenario, MRCP was cost effective only at a cost of $40 or less, although increasing the ECRP complication cost to $20,000 allowed MRCP's cost to rise to $230.

"In general, as the pretest probability goes up, the cost of the test has to drop down," Tello said. "So if you're getting more and more suspicious that there's disease, you shouldn't be doing a noninvasive test. As the complication rate goes up, then you're allowed a little more leeway in how expensive the test can be, because you're able to save patients from having complications they should never have had to begin with."

The cost of the noninvasive test can also rise as the cost of misdiagnosis rises, he said, because the noninvasive test prevents complications from procedures the patient shouldn't have been exposed to.

In summary, if the gold standard test costs more than $1,000, and the complication rate is greater than about 5%, the pretest disease probability must be low -- probably less than 0.03 -- for a noninvasive test to be cost-effective, Tello concluded. Depending on these key variables, the noninvasive procedure may not be warranted.

For simplicity's sake, he said, the study variables were based on estimates of hospital costs, intentionally sidestepping larger ethical questions such as the value of lost work, disability, and even life itself.

"These questions can walk you into a morass in which you can never come up with numbers, and never come up with an answer," he said. "My response, however, is that I'm basically setting limits. If we use just the crudest estimates of hospital costs, then that gives me an upper boundary for which we can use virtual colonoscopy."

"If we turn around and say, well, a perforated colon is going to cost $3,000 because he's got to stay in the hospital a couple of days, and one of the days is in the ICU -- that's a minimum context," he explained. "That will give you an upper limit that calculates out to 'You better not do virtual colonoscopy for more than $727.' If your technical and professional fee works out to ... $1,500, then it's not cost effective to be doing that."

Tello said the method could be helpful in cases where physicians or departments cling to certain procedures regardless of the numbers. And, he added, the formulas offer a handy way of demonstrating a test's efficacy or lack thereof to hospital administrators.

"We'd like to have this programmed on a [hand]held computer, so if someone at your institution asks 'What's the pretest probability?' you'll have the appropriate option," Tello said. "Of course, then you're going to have to deal with your colleagues who really want to do noninvasive testing for their own motivations, so it's going to be a very interesting sort of evolution of this issue."

By Eric Barnes staff writer
June 7, 2001

Click here to post your comments about this story. Please include the headline of the article in your message.

Copyright © 2001

Page 1 of 179
Next Page