How to do it: Medical testing edition

I was commenting just recently on the cult of big ideas, where people whose life experiences have given them hierarchical power are suckers for “ideas” that are mostly blather, lots of words about the irrelevant bits of the problem, distracting attention from the real difficulties. And now Theranos is in the news. I read about this company, started by the obviously charismatic Elizabeth Holmes, in The New Yorker about a year ago. My immediate reaction was, this must be a joke. It was very much in the spirit of Monty Python’s How to do it.

Theranos, a Silicon Valley company[…], is working to upend the lucrative business of blood testing. Blood analysis is integral to medicine. When your physician wants to check some aspect of your health, such as your cholesterol or glucose levels, or look for indications of kidney or liver problems, a blood test is often required. This typically involves a long needle and several blood-filled vials, which are sent to a lab for analysis… [Theranos] has developed blood tests that can help detect dozens of medical conditions, from high cholesterol to cancer, based on a drop or two of blood drawn with a pinprick from your finger. Holmes told the audience that blood testing can be done more quickly, conveniently, and inexpensively, and that lives can be saved as a consequence.

Sounds wonderful. Quick. Convenient. Inexpensive. Saving lives. How is she going to do all that? Well, she wears “a black suit and a black cotton turtleneck, reminiscent of Steve Jobs”. She dropped out of Stanford. She has a board of directors full of highly influential aged former politicians, but no scientists, so far as I can tell. She “is in advanced discussions with the Cleveland Clinic. It has also opened centers in forty-one Walgreens pharmacies, with plans to open thousands more. If you show the pharmacist your I.D., your insurance card, and a doctor’s note, you can have your blood drawn right there…. A typical lab test for cholesterol can cost fifty dollars or more; the Theranos test at Walgreens costs two dollars and ninety-nine cents.”

But then you wonder (at least, I wondered), given that it sounds obviously better, doing tests on a drop or two of blood for a tenth of the cost, why isn’t anyone else doing it? What is she doing differently, and how do we know it works? My immediate presumption was, they wouldn’t be reporting on it unless there were very substantial confirmation that the methods work, and extensive scientific support for her brilliant new techniques.

But no.

[Theranos] occupies a regulatory gray area. Most other diagnostic labs, including Quest and Laboratory Corporation of America, perform blood tests on equipment that they buy from outside manufacturers, like Siemens and Roche Diagnostics. Before those devices can be sold, they must be approved by the F.D.A., a process that makes their tests’ performances more visible to the public. But, since Theranos manufactures its own testing equipment, the F.D.A. doesn’t need to approve it, as long as the company doesn’t sell it or move it out of its labs. Holmes said that the company has long resisted discussing how its technology works or how it makes money in order to avoid tipping off potential competitors.

Right. The entire evidence that Holmes was able to send to the journalist about the reliability of her novel methods was a single peer-reviewed pilot study. On the other hand, they did obtain certification in 48 states, requiring periodic evaluation of the reliability of their results, so that sounds pretty convincing…

Until you read the new report in the Wall Street Journal:

Theranos also hasn’t disclosed publicly that it does the vast majority of its tests with traditional machines bought from companies like Siemens AG.

Well, that would explain how they get certifiably correct results.