Proving breast cancer screening efficacy remains complex

Jan 9, 2003

Editorial opinion by Dr. Daniel Kopans

I was surprised and concerned to learn that the Radiology article on ultrasound screening for breast cancer by Dr. Thomas Kolb and colleagues had won The Minnies award for Scientific Paper of the Year ("Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: An analysis of 27,825 patient evaluations," October 2002, Vol.225:1, pp. 165-175).

My concern does not demean the obvious effort that went into this study, which has now been updated three times. My concern lies in the fact that the study design precludes use of the results in any meaningful scientific fashion. Despite the unconscionable hype that accompanied the publication of this paper, the results do not prove the efficacy of ultrasound screening. There are still no data that show any benefit from detecting cancers by ultrasound alone.

The past few decades have seen medicine evolve from a reliance on anecdote and experience to "evidence-based" science. Radiologists have been criticized for being slow to adopt the scientific approach. The Kolb article is the ultimate anecdotal study, which uses very large numbers with little scientific validity, despite its being sprinkled with scientific terminology (statistical significance, etc.). The fact that it was published in a respected journal is an example of the fact that peer review does not always guarantee the validity of a paper.

It is surprising, however, that the editors of Radiology are clearly unfamiliar with the requirements for proving the efficacy of a screening test. Not only did they approve the article for publication, but it was irresponsibly promoted in the media with a press release and a press conference.

As a result, women and their physicians may have been misled into believing that screening for breast cancer with ultrasound had been shown to be efficacious when the study does not show this. Nor was it properly designed and performed to be able to draw this conclusion.

There are basic principles and problems involved in demonstrating the efficacy of a screening test that must be addressed. Contrary to what the authors wrote in their paper, I did not devise these requirements. They have evolved over time as experience and understanding has accumulated. There are many pitfalls in determining whether or not a screening test is efficacious, and these cannot be ignored.

Since screening for breast cancer is an effort to defer or prevent death, a test must be shown to be able to delay or decrease deaths in order to be efficacious. This requires trials that are similar in design to those that test the efficacy of a chemotherapeutic agent. The only exception would be if factors were to be identified to directly predict who will die and who will survive. Unfortunately, none of these "surrogate end-points" have been found for breast cancer.

Although the size and stage of breast cancers are fairly good predictors of whether a patient will survive or be killed by the malignancy, they are far from directly predictive. Thus, merely finding smaller, earlier-stage cancers does not guarantee any benefit. The issues are complex, but clear. I would direct the interested reader to my textbook, Breast Imaging (Lippincott Williams & Wilkins, Philadelphia, 1997), for a more detailed discussion as to what is needed for a screening test to be efficacious.

If every single cancer were lethal, then the problem would be much simpler. Fortunately, most women who develop breast cancer do not die from it. Since there are clearly breast cancers that are not lethal (unfortunately, we cannot, as yet, tell which will be and which will not be), and there are other breast cancers that have metastasized before any test can detect them, then just finding cancers earlier may be of no benefit. In fact, it could be harmful.

For example, if every cancer that was found by ultrasound alone was of the nonlethal variety, then finding these cancers would lead to treatment that was not only unnecessary, but would be harmful (lumpectomies, mastectomies, radiation and chemotherapy, etc.).

The same would be true if it only found tumors that had already metastasized because these are uniformly fatal despite treatment. The only way to show that a breast cancer screening test is efficacious is through randomized, controlled trials. Large groups of women are randomly divided into two identical groups. One group is offered the new screening test and the other has the usual healthcare.

If the trial is performed properly, and the test significantly reduces deaths in the screening group, then the test is efficacious. No one has devised a way to get around this requirement since there are so many potential influences that can bias results using any other approach. It is the crucial requirement that a breast cancer screening test show decreased mortality in randomized, controlled trials that has kept the controversy boiling around mammography screening.

Ultrasound screening is no exception. Until some "surrogate" endpoints can be identified that accurately predict lethality, a decrease in deaths is the only acceptable endpoint.

Finally, the Kolb data likely exaggerate the accuracy of ultrasound since, as the authors admit, the examiner was not blinded to the mammography results. Ultrasound alone may have overlooked lesions if not for the mammogram directing and biasing the radiologist. The authors' suggestion that ultrasound is not used in a vacuum, but in conjunction with mammography, is an excuse for having not performed the study correctly.

The only way to know the individual contribution of a test is to perform that test without any other information. Once the results from this "blinded" study are collected, then the study can be redone using the additional information to see how the results change when using the combined information.

Unfortunately, the Kolb paper is not what many seem to think it is. In fact, it is a dangerous paper. Uninformed readers will misconstrue its results. It does not prove any benefit from ultrasound screening for breast cancer. Not only is ultrasound screening not the standard of care, but given the potential harm that might result from ultrasound screening (biopsies, unnecessary treatment, etc.), ultrasound screening of the breast should only be performed in properly designed scientific trials where the patient has given informed consent.

By Dr. Daniel Kopans
AuntMinnie.com contributing writer
January 10, 2003

Dr. Kopans is the director of breast imaging at Massachusetts General Hospital in Boston and a professor of radiology at Harvard Medical School.

In deference to Dr. Kolb’s wishes, we have postponed publication of his rebuttal to Dr. Kopans until it is first published in Radiology, scheduled for later this spring.

Mammographers question newspaper’s ‘crusade’ against breast imaging, June 28, 2002

The opinions expressed in guest editorials are those of the author, and do not necessarily reflect the views of AuntMinnie.com.