Antidotes to Anecdotes

Did you know that eating cookies is a dangerous practice? At least that is what the British press is telling us after a cookie-manufacturer-sponsored report showed that cookies can lead to burns when one dunks them, back strain when one picks up a crumb, and doctor visits for crumbs that end up where they shouldn’t (don’t think about this one too hard). Of course, the competition’s brand of cookie was far more “dangerous” than that of the company doing the research.

According to Kathleeen Fairman, associate editor at the Journal of Managed Care Pharmacy, this rather silly example demonstrates what often happens in assessments made by managed care decision makers. They need to sift through vast quantities of information: peer-reviewed  studies, press reports, and anecdotal evidence from blogs and the Internet. The normal policy would be to give the most weight to peer-reviewed literature, but those also suffer from problems, not the least of which being that the research is usually funded by the drug-manufacturer. Thus, even peer-reviewed literature can become extremely misleading.

Fairman goes on to show how a Pfizer-sponsored study claimed to show that use of Lipitor is more cost-effective than use of the generic simvastatin, simply because Lipitor gives clinically superior results. However, her analysis of their data pretty convincingly shows that this conclusion was, at the very least, questionable. Lipitor is not significantly better than simvastatin, even though it costs four to five times as much.

Similarly, she analyzes claims that Plavix + aspirin is marginally better than placebo (nothing) + aspirin in improving outcome for patients with cardiovascular risk factors. But, an analysis of the statistical methods used in the CHARISMA study reveals that fundamental errors were made and the results were not statistically significant. In other words, the CHARISMA study, when properly analyzed, effectively demonstrates thatPlavix is as good as nothing, but not much better.

Fairman concludes that it is vital to our health and financial well-being that clinicians and managed care organizations have access to “evidence-based information” and valid research results. She also recommends “healthy skepticism about commonly accepted practices and ideas in health care.” In other words, we should base our science on impartial evaluation of evidence, not mere consensus (or claims of statistical significance).