Among the reaction to my Atlantic article there has of course been a certain amount of skepticism and criticism. Some of the criticism has been perfectly fair and insightful, pointing out the ways in which I or Ioannidis (the physician-researcher I profile in the article who has documented and analyzed the high wrongness rate in medical research) might ourselves be biased, and might be spinning some aspects of the story. I touch on this problem in the article, and have a whole chapter on it in the book, and I have no interest in denying that my own bias and sloppiness (as well as those of the people I interview and those of my editors) may skew things.
I generally welcome criticism, and usually don’t respond to it–I figured I had my say–except as part of a formal response to comments submitted to the publication that ran the article. But I saw a blog post today that I think calls for a bit of a response. The post, by the physician-researcher David Gorski (whom I actually briefly quote in the Atlantic article), echoes posts in other blogs from a few people in or close to the medical community in essentially suggesting that the problems with research that Ioannidis has uncovered and that I report on in the article aren’t really big problems; rather, they’re just an acceptable part of the nature of research. I don’t think so. Bias and sloppiness may be routine and perhaps inevitable even among top researchers, but that doesn’t mean we shouldn’t be made aware of the extent of the problems and the toll it takes on the credibility of medical findings–and the vast majority of researchers seem to agree, to judge by most of what has been posted in response to the article. Gorski’s post also states that we should focus on the fact that (as I report in the article) 90 percent of large randomized controlled trials tend to hold up, and not pay much attention to the fact that other types of studies sink to levels as low 10 percent rightness and even lower. This is to me a shocking argument, considering that large RCTs make up a tiny percentage of the studies that fill journals, make headlines and influence treatment and lifestyle decisions.
Finally, the post argues that everyone should put their energy not into looking into the problems with mainstream medical research, but rather into the poor or non-existent science behind alternative medicine–and that we should keep our mouths shut about whatever problems we do find with mainstream medicine because it only gives ammunition to the alternative medicine crowd. Once again, I have to strongly disagree. People who are drawn to alternative medicine have already proven themselves essentially either uninterested in or incapable of assessing scientific evidence, and when researchers like Gorski rant and rave about alternative medicine they’re really just preaching to the choir, as is clear from the comments that appear under his post. In fact, compare the nearly unanimous point of view in these comments to those that appear almost anywhere else in response to the Atlantic article–Gorski’s fans clearly have their minds made up about people who criticize mainstream medical research, and they don’t even actually have to read the criticism themselves to feel comfortable commenting dismissively and authoritatively on it. Some of the comments, following Gorski’s lead, include ad hominem attacks on me that are based entirely on sloppy and mostly incorrect assumptions about my background.
Unlike the well-known absence (with a few exceptions) of good science behind alternative medicine, the serious problems with mainstream medical research have largely been unknown outside of the medical community itself. I believe the public has a right to hear about them, and to judge by the reaction the public seems quite interested in hearing about them. I quote Ioannidis in the article as pointing out that if mainstream medical science tries to keep quiet about its problems and limitations, then it is doing what it accuses alternative medicine of doing–misleading the public. I’m glad Gorski and his fans represent a small minority of the medical community in being unwilling to own up to and communicate these problems–see, for example, what the British Medical Journal had to say about my book–and in focusing instead on endlessly recycling the same old complaints about alternative medicine because it makes them look good in comparison.
And for the record: Along the way, Gorski belches out the sarcasm-and-bile-drenched claim that my earlier post on this blog relating to quack autism researcher Andrew Wakefield reflects my refusal to recognize that Wakefield was wrong. That’s just plain silly. The point of the post was that the outing of a rare, gross fraud like Wakefield distracts from the more widespread, routine problems in medical research trustworthiness. I think I was pretty clear about that, but judge for yourself. I happen to be a little sensitive to the suggestion that I support quack autism research, because two members of my immediate family work with children with autism, relying solely on the treatment that is the mainstream standard of care, applied behavior analysis. I know quite a bit about autism quackery, actually. I just didn’t think the intelligent readers of this blog needed me to rant and rave, Gorski-style, about an obvious and blatant charlatan who had long been making global headlines for his misdeeds. (And my Wakefield post linked to an article that described the misdeeds at length, though I just now changed the link to point to a similar article because the original is no longer online.) Considering that he passes himself off as the champion of objectivity, facts and reason, Gorski seems surprisingly comfortable distorting the facts to fit his nearly undisguised biases. But maybe we should be grateful for undisguised biases–it’s really the well-disguised ones we need to watch out for.
David H. Freedman 
I read a David Gorski post in response to your article, here: http://www.sciencebasedmedicine.org/?p=7734In it he says that bias is important and designing well-designed studies is essential to minimize the effect of bias. He said that scientists are aware of the problems, which is why they reserve judgement on the results of small, poorly designed studies. What he said about Wakefield was, “One wonders, given how much he distrusts the medical literature, Freedman actually knew Wakefield was wrong. After all, all the studies that refute Wakefield presumably suffer from the same intractable problems that Freedman sees in all medical literature.”You are clearly responding to a very different blog post. Which is the one you read?
Thanks for writing, Alison. You're being sarcastic in pretending confusion, I assume, since I see you're a frequent and extremely supportive comment-contributor to Gorski's blog, and have even commented on this very post there. ("Silly" was your exact term.) I hope you'll comment here often as well, even if it is less supportively, and even if you choose to employ sarcasm, the apparent medium of choice for Gorski and his sometimes contentious fans.Anyway, you seem to be implying that I did not fairly represent Gorski's comments. It seems to me that you yourself provide in your comments the Gorski quote that clearly suggests I don't recognize the extremity of Wakefield's misdeeds, so I'm not sure what your objection is in that case. As for the Gorski points that you paraphrase, it is not clear to me how they prove that Gorski didn't say the things I say he did. It's true, Gorski is himself an outspoken critic of weak studies, and much of what he says, as the paraphrased comments you carefully chose indicate, are absolutely in keeping with what the Atlantic article points out. But that doesn't alter the fact that he says exactly what I say he says in his post. I leave it to readers to look at his long, angry, mocking, derisive post (which I link to in this post, a link which you thoughtfully re-provide) and judge for themselves, rather than judging by a few paraphrased snippets that are apparently intended to mislead readers into thinking Gorski is the very voice of reason with regard to the trouble with mainstream studies, and that he doesn't have it in for my article's claims that mainstream medicine has serious problems that need to be pointed out to the public.
I’m not sure that David Gorski would characterize me as “extremely supportive.” Frequent, yes. Probably also “cranky and argumentative.”I cited David Gorski wondering how you think you know that any given published results are wrong. Presumably you think the published results that discredit Andrew Wakefield have more credibility than Andrew Wakefield’s published results. This suggests you believe it possible to preferentially focus on higher-credibility work. David Gorski wasn’t nice to you, and his criticism of you for being “inflammatory” is hypocritical, as noted. But no, I don’t see that he said the things you say he said. At all.
Hey, quit sounding so reasonable, you're making me look bad.I absolutely believe it's possible to preferentially focus on higher-credibility work, and indeed that's really one of the main points of my whole "Wrong" project. When it's revealed that someone like Wakefield committed outright fraud, for example, that certainly makes it easier. I'm no fan of pseudo-science, bad science, alternative science or non-science. I actually say quite a bit in my book about the issue of focusing on higher-credibility work. As long as the Atlantic piece was, it felt to me like I had to leave out 95% of what I wanted to say, including that issue.As for our apparent disagreement on what Gorski did or didn't say in his post, I declare respect for your opinion, and I encourage readers to read Gorski's post for themselves–it's a good read!–and then to consider writing in to share their impressions.
Please accept my compliments for your "Atlantic" article on "Lies, Damned Lies, and Medical Science" in which you flesh out and provide context to Dr. Ioannidis findings about the high error rate in medical research. When wagon-circling medical critics such as Dr. David Gorski claim that this is not a big problem but an acceptable part of medical research, I suggest you ask them how acceptable they find it that many tens of thousands of premature babies around the world have been and still are being killed or severely brain- damaged and blinded over more than a half century of alleged research about retinopathy of prematurity (ROP), mostly as the result of two blatantly rigged US clinical trials, both "Platinum Standard" multi-hospital and randomized, and both deliberately deceptive. You will find a detailed and fully documented account of these research frauds, and of their uncritical acceptance by a gullible and not even remotely science-based medical community, in my article "Fake Science and Bogus Bioethics in Medical Research about Premature Babies" athttp://retinopathyofprematurity.org/01summary.htm. Similarly, Dr. Gorski claims on his blog atsciencebasedmedicine.org about the ethics violations in medical research, such as the recently outed syphilis studies in Guatemala, that he will not "turn a blind eye when such lapses occur in trials of ostensibly science-based medicine and nor should anyone else." Good for him and for those safely long past transgressions, but many other medical doctors firmly close their eyes to more recent lapses and even help to cover them up. Right while President Clinton apologized publicly to the victims of the Tuskegee abuses, the designers of the rigged LIGHT-ROP study of baby-blinding by nursery light committed gross ethics violations against their captive patients very similar to those in Guatemala and Tuskegee. The New England Journal of Medicine published that study in 1998 although its editors had been alerted to the scientific frauds and ethics abuses. They, as well as the IRBs, the National Eye Institute, and the US government agencies for "Patient Protection" and for "Research Integrity", simply ignored those openly prescribed violations. I posted a description of these recent lapses at http://retinopathyofprematurity.org/Re-Tuskegee.htm. The scientific frauds and patient abuses in this fake baby- blinding research still continue to harm many preemies every day. I don't believe this is an acceptable part of medical research. You clearly don't think so either, but I am curious whether Dr. Gorski or anyone else in the medical community will speak out against these flagrant lapses or continue to turn a blindeye towards them.
Dear David, great article! I have addressed some of the issues you raise on my blog as well. And I too have felt the blunt instrument of Gorsky's sarcasm, as you may have seen on his Insolence blog. In fact, as you can see from many of my blog posts, I am a great believer in the nuanced conversation that our imprecise science deserves. So, keep up the dialogue!
Thanks for writing in Marya. I did see your exchange with Gorski, and good for you! (And I credit Gorski for publishing the exchange as well.) Gorski is properly an outspoken critic of bad studies, but his insistence on focusing only on certain, easy-to-criticize types of studies and fields tends in my opinion to distract from some of the larger problems with research, and his sometimes angry and disdainful attitude is not helpful. Here's to more conversation on the subject!
And by the way, thanks to Peter for writing in as well. (His comment is up above a little ways.) He actually emailed me the comment directly because the blog software wasn't publishing his comment, and I posted the comment for him and wrote him back directly–I don't want anyone to think I'm ignoring commentors. I encourage anyone who's having trouble posting a comment to just send it to me as per the MSOMed "about" page. (The blog software seems to choke on some longer comments, but I welcome them!)
Hear, hear, David! I am all for more conversation and shining the light on the uncertainties. Just did a post inspired by your titular question.http://evimedgroup.blogspot.com/2010/11/manipulating-science.html
David Gorski is, in my opinion, the worst kind of skeptic. He professes to abhore anything based on unsound science, but approaches the mainstream vs. alternative medicine debate with a preconception that completely precludes him from reaching a nuanced and comprehensive understanding of the issues at hand. Kudos to you Mr. Freedman.