Writing in the NY Times, management professor Clifton Leaf quotes (apparently with approval) comments that ought to win the GlaxoSmithKline Prize for Self-Serving Distortions by a Pharmaceutical Company. Referring to the prominent recent failure of Genentech’s cancer drug Avastin to prolong the lives of patients with glioblastoma multiforme, Leaf writes
Doctors had no more clarity after the trial about how to treat brain cancer patients than they had before. Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average. But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)
Indeed, even after some 400 completed clinical trials in various cancers, it’s not clear why Avastin works (or doesn’t work) in any single patient. “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not,” says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant Roche, which makes the drug.
This is, in technical terms, a load of crap, and it’s exactly the sort of crap that double-blind randomised clinical trials are supposed to rescue us from. People are generally prone to see patterns in random outcomes; physicians are probably worse than the average person, because their training and their culture biases them toward action over inaction.
It’s bizarre, the breezy self-confidence with which Leaf (and the Genentech spokesman) can point to a trial where the treatment group did worse than the placebo group — median survival of 15.7 months vs. 16.1 months — and conclude that the drug is helping some people, we just can’t tell which they are. If there are “responders”, who do better with Avastin than they would have otherwise, then there must also be a subgroup of patients who were harmed by the treatment. (If the “responders” are a very small subset, or the benefits are very small, they could just be lost in the statistical noise, but of course that’s true for any test. You can only say the average effect is likely in a certain range, not that it is definitely zero.)
It’s not impossible that there are some measurable criteria that would isolate a subgroup of patients who would benefit from Avastin, and separate them from another subgroup that would be harmed by it. But I don’t think there is anything but wishful thinking driving insistence that there must be something there, just because doctors have the impression that some patients are being helped. The history of medicine is littered with treatments that physicians were absolutely sure were effective, because they’d seen them work, but that were demonstrated to be useless (or worse) when tested with an appropriate study design. (See portacaval shunt.)
The system of clinical trials that we have is predicated on the presumption that most treatments we try just won’t work, so we want strong positive evidence that they do. This is all the more true when cognitive biases and financial self interest are pushing people to see benefits that are simply not there.
Common Infirmities 