David H. Freedman

The diet component of the program consisted of a nutritionally adequate, low-fat (20%-30% of energy), reduced-energy diet (typically 1200-2000 kcal/d) that included prepackaged prepared food items with increased amounts of vegetables and fruits to reduce the energy density of the diet. The approach was tailored so that participants could choose regular foods when preferred. Participants were encouraged during the initial period to follow a menu plan with prepackaged foods, which would provide 42% to 68% of energy for those who choose not to deviate from the plan. Regular foods, such as vegetables, fruit, cereal or grain products, low-fat dairy products, lean meat or the equivalent, and unsaturated fat sources were recommended to achieve the total prescribed energy intake. Over time, participants were transitioned to a meal plan based mainly on food not provided by the commercial program, although participants could choose to include 1 prepackaged meal per day during weight loss maintenance. Prepared foods and counselors were provided by Jenny Craig Inc (Carlsbad, California).

Increased physical activity was another program component; the goal was 30 minutes of physical activity on 5 or more days per week. Program material and counseling addressed attitudes about weight, food, and physical activity and included recipes and guidance for eating in restaurants, CDs and DVDs to increase physical activity, and online tools and support.

By study end, more than half in either intervention group….had a weight loss of at least 5% compared with 29%…of usual care participants….  More than twice the proportion of participants in the center-based and telephone-based intervention groups compared with participants in the usual care group…had a weight loss of 10% or more of baseline weight at 24 months….

Nicholas Bakalar has a good article about physicians’ efforts to encourage patient weight loss in the New York Times “Science Times” section. Bakalar is one of the most careful writers at the Times when it comes to avoiding flashy medical findings that aren’t likely to hold up, as well in making a point of looking for and clearly reporting on the limitations of the studies he covers. You wouldn’t think those would be unusual traits in science journalists, and especially in Times reporters, but they are. (I should also mention I’ve met Bakalar a few times and consider him a friend, but my admiration for his work pre-dates my knowing him.)

Bakalar’s piece makes two important points: physicians normally have little luck in getting patients to lose weight, and physicians tend to have more impact when instead of trying to push a patient into losing weight they instead work with the patient to try to figure out together what to do about the problem. I’ve been researching both of these issues for my ongoing obesity projects, and Bakalar’s article is spot on with regard to both of them. I’ve asked dozens of physicians how many patients they’ve managed to convince to lose weight, and the answer is pretty much always just about zero. And the idea of working with patients to come up with an appropriate plan that focuses in part on helping patients to recognize and deal with their lack of motivation, and in part in figuring out what actions can be taken that are realistic for whatever level of motivation they have, is a critical part of a behavioral approach to weight loss. Something I’ll be saying a lot more about in this blog and elsewhere is the fact that the behavioral approach, while backed by a lot of evidence, made famous by Weight Watchers and pushed by many highly credible experts and public health officials, is largely ignored by most of the overweight public as well as by most physicians and obesity researchers–and I think it’s a big reason we keep getting bigger.

Separately, here’s a piece, this one in Canada’s Globe and Mail, that hits on a another point I’ve become very interested in: the terrible job that people and devices do in measuring calories burned when exercising.  I’ll have more to say about this soon.

Among the reaction to my Atlantic article there has of course been a certain amount of skepticism and criticism.  Some of the criticism has been perfectly fair and insightful, pointing out the ways in which I or Ioannidis (the physician-researcher I profile in the article who has documented and analyzed the high wrongness rate in medical research) might ourselves be biased, and might be spinning some aspects of the story.  I touch on this problem in the article, and have a whole chapter on it in the book, and I have no interest in denying that my own bias and sloppiness (as well as those of the people I interview and those of my editors) may skew things.

I generally welcome criticism, and usually don’t respond to it–I figured I had my say–except as part of a formal response to comments submitted to the publication that ran the article. But I saw a blog post today that I think calls for a bit of a response.  The post, by the physician-researcher David Gorski (whom I actually briefly quote in the Atlantic article), echoes posts in other blogs from a few people in or close to the medical community in essentially suggesting that the problems with research that Ioannidis has uncovered and that I report on in the article aren’t really big problems; rather, they’re just an acceptable part of the nature of research.  I don’t think so.  Bias and sloppiness may be routine and perhaps inevitable even among top researchers, but that doesn’t mean we shouldn’t be made aware of the extent of the problems and the toll it takes on the credibility of medical findings–and the vast majority of researchers seem to agree, to judge by most of what has been posted in response to the article.  Gorski’s post also states that we should focus on the fact that (as I report in the article) 90 percent of large randomized controlled trials tend to hold up, and not pay much attention to the fact that other types of studies sink to levels as low 10 percent rightness and even lower.  This is to me a shocking argument, considering that large RCTs make up a tiny percentage of the studies that fill journals, make headlines and influence treatment and lifestyle decisions.

Finally, the post argues that everyone should put their energy not into looking into the problems with mainstream medical research, but rather into the poor or non-existent science behind alternative medicine–and that we should keep our mouths shut about whatever problems we do find with mainstream medicine because it only gives ammunition to the alternative medicine crowd.  Once again, I have to strongly disagree.  People who are drawn to alternative medicine have already proven themselves essentially either uninterested in or incapable of assessing scientific evidence, and when researchers like Gorski rant and rave about alternative medicine they’re really just preaching to the choir, as is clear from the comments that appear under his post.  In fact, compare the nearly unanimous point of view in these comments to those that appear almost anywhere else in response to the Atlantic article–Gorski’s fans clearly have their minds made up about people who criticize mainstream medical research, and they don’t even actually have to read the criticism themselves to feel comfortable commenting dismissively and authoritatively on it.  Some of the comments, following Gorski’s lead, include ad hominem attacks on me that are based entirely on sloppy and mostly incorrect assumptions about my background.

Unlike the well-known absence (with a few exceptions) of good science behind alternative medicine, the serious problems with mainstream medical research have largely been unknown outside of the medical community itself.  I believe the public has a right to hear about them, and to judge by the reaction the public seems quite interested in hearing about them.  I quote Ioannidis in the article as pointing out that if mainstream medical science tries to keep quiet about its problems and limitations, then it is doing what it accuses alternative medicine of doing–misleading the public.  I’m glad Gorski and his fans represent a small minority of the medical community in being unwilling to own up to and communicate these problems–see, for example, what the British Medical Journal had to say about my book–and in focusing instead on endlessly recycling the same old complaints about alternative medicine because it makes them look good in comparison.

And for the record: Along the way, Gorski belches out the sarcasm-and-bile-drenched claim that my earlier post on this blog relating to quack autism researcher Andrew Wakefield reflects my refusal to recognize that Wakefield was wrong.  That’s just plain silly.  The point of the post was that the outing of a rare, gross fraud like Wakefield distracts from the more widespread, routine problems in medical research trustworthiness.  I think I was pretty clear about that, but judge for yourself.  I happen to be a little sensitive to the suggestion that I support quack autism research, because two members of my immediate family work with children with autism, relying solely on the treatment that is the mainstream standard of care, applied behavior analysis.  I know quite a bit about autism quackery, actually.  I just didn’t think the intelligent readers of this blog needed me to rant and rave, Gorski-style, about an obvious and blatant charlatan who had long been making global headlines for his misdeeds. (And my Wakefield post linked to an article that described the misdeeds at length, though I just now changed the link to point to a similar article because the original is no longer online.) Considering that he passes himself off as the champion of objectivity, facts and reason, Gorski seems surprisingly comfortable distorting the facts to fit his nearly undisguised biases.  But maybe we should be grateful for undisguised biases–it’s really the well-disguised ones we need to watch out for.

http://www.msnbc.msn.com/id/32545640

How does a legendary retailer deal with irate customers in the age of user reviews?

From my “Tech Support” channel on The New York Times‘ “You’re the Boss” blog:

A camera-store salesman recently steered me away from the compact, ultrahigh-tech camera I thought I wanted. Smaller is fine, he told me, but only if it fits in your pocket — any bigger will end up hanging from your shoulder anyway, so what’s the advantage? And don’t buy based on fancy features, he added; the differences between comparable models rarely result in better pictures for amateurs. The one feature you’ll really appreciate, he said, is how the camera feels in your hand.
Simple insights, but they pointed me to a very different camera, one that I’m deliriously happy with. Not only that, it was a few hundred bucks cheaper than the one I had wanted. Maybe that’s why everyone at the five other camera stores I had been to was a lot more enthusiastic about the compact model…..read more

How even top-shelf science ends up leading us astray

From Kayla Webley’s Q&A with me on Time.com:

To read the factoids David Freedman rattles off in his book Wrong is terrifying. He begins by writing that about two-thirds of the findings published in the top medical journals are refuted within a few years. It gets worse. As much as 90% of physicians’ medical knowledge has been found to be substantially or completely wrong. In fact, there is a 1 in 12 chance that a doctor’s diagnosis will be so wrong that it causes the patient significant harm. And it’s not just medicine. Economists have found that all studies published in economics journals are likely to be wrong. Professionally prepared tax returns are more likely to contain significant errors than self-prepared returns. Half of all newspaper articles contain at least one factual error. So why, then, do we blindly follow experts? Freedman has an idea, which he elaborates on in his book Wrong: Why Experts Keep Failing Us — and How to Know When Not to Trust Them. Freedman talked to TIME about why we believe experts, how to find good advice and why we should trust him — even though he’s kind of an expert.

Time: You say that many experts are wrong, yet you quote many experts in your book. Are these experts wrong too?

DHF: They very well may be, but these are people who study expertise. They know how other experts go wrong because this is what they study, so maybe they’re better at avoiding some of these problems. Maybe they’re a little more careful with their data and they work a little harder to not mislead people…..read more

Why researchers look for answers where the looking is good, rather than where the answers are hiding

From my article in the July 2010 issue of Discover:

A bolt of excitement ran through the field of cardiology in the early 1980s when anti-arrhythmic drugs burst onto the scene.  Cardiologists knew that heart-attack victims with steadier heartbeats were far more likely to survive, so a drug that could tamp out heartbeat irregularities seemed like a no-brainer.  The drugs quickly became the standard of care for heart-attack patients, and were soon smoothing out heartbeats in intensive-care wards around the US, as described in numerous published studies.  But in the early 1990s cardiologists realized the drugs were also doing something else: killing about 40,000 heart-attack patients a year.  Yes, the hearts were beating more regularly on the drug, but the patients were on average one-third as likely to pull through.  Cardiologists had been so focused on immediately measurable heartbeat irregularities that they hadn’t been paying enough attention to the longer-term, but far more important, variable of death.
   There’s an old joke scientists love to tell: A police officer finds a drunk man late at night crawling on his hands and knees on a sidewalk under a streetlight.  Questioned, the drunk man tells her he’s looking for his wallet.  When the officer asks if he’s sure that he dropped the wallet here, the man replies that he thinks he more likely dropped it across the street.  Then why are you looking over here? asks the befuddled officer.  Because the light’s better here, explains the drunk man.
   That drunk fellow is in good company.  Many and possibly most scientists spend their careers looking for answers where the light’s better rather than where the truth is more likely to lie….read more

Is John “Grizz” Deal the greatest salesperson around?

From my article in the April 2010 issue of Inc. Magazine:

You can tell a lot about what you’re up against in a sales pitch by the way they serve you coffee,” John Deal mumbles to me, as the others in the room noisily take their seats around the conference table at a well-known British engineering and defense contracting company on a dreary day in central England. I take this to mean that Deal has his work cut out for him, given that his prospects have unceremoniously plunked down in front of him a jug of scalding coffee and a stack of plastic cups, with no cream or sugar in sight….. read more

When the human genome was sequenced a decade ago, the world lit up with talk of new treatments that would help us cheat death.  So why do exercise and healthy eating still do more for us than doctors can?

From my article in the November 2009 issue of Fast Company:

Ernest Hemingway’s writing may have tended to the short and sharp, but the man himself was apparently fond of the cuddly and extraneous, at least when it came to kittens with too many toes. A sea-captain friend of Hemingway’s, it seems, persuaded him to take in a polydactylic cat, and that cat became the progenitor of a colony of overly toed felines thriving today in and around the museum in Key West that was Hemingway’s home. The patterns of inheritance among those cats have even helped shed a bit of light on certain defects in human DNA. And so it is that Papa retroactively became an early contributor to the science of the human genome.
   I learn this from Nadav Ahituv, a rising-star geneticist at the University of California, San Francisco, Medical Center, who studies the genetic roots of limb-related defects, obesity, and drug absorption….. read more

The scientists at Emotiv have made a brain-wave-reading headset that lets you conjure entire worlds using nothing but your mind.  Now comes the hard part. 

From my cover story in the December 2008 issue of Inc. Magazine:

I’m sitting in a darkened room, attempting to move a large block with nothing but my thoughts. Move, damn you; I am your master. After a long moment, the block trembles a bit, then slowly skids toward me a few feet before stopping.
   Brain waves usually are monitored in hospitals or research labs, but I’m in a conference room at a company called Emotiv, where a few dozen scientists have developed a headset and software that quite literally reads my mind, allowing me play a sort of video game with nothing but sheer thought.  For $299, you and yours will very soon be able to vaporize onscreen enemies with an angry thought, have your online characters smile when you smile, and see video games react to your level of excitement. And that’s just for starters. Backed by some impressive partners, Emotiv has a long-range strategy that sounds like a business-school case study from the 22nd century….  read more

A global effort is underway to invent a better way of finding things on the Web. Could Google be vulnerable?

From my cover story in the March 7, 2007, issue of Newsweek (International Edition):

….Despite spending billions trying to diversify beyond the straightforward search offered on its stripped-down, almost childlike home page, Google reaps about 60 percent of its outsized revenues and more than 80 percent of its profits from ads on that page, according to analysts’ estimates. That means the company’s success continues to hinge on the dominance of its simple search. There are no guarantees that its dominance will last. It is threatened by a massive worldwide effort to build a better search, involving giant high-tech rivals, governments in Europe and Asia, and hundreds of tiny start-ups founded by academic wunderkinders much like Sergey Brin and Larry Page, the Stanford graduate students who founded Google in 1998. And it’s also dependent on an online public that may make up the most fickle market in history, an audience whose interests are already showing signs of wandering outside the search box…..  read more

By Penelope Green, The New York Times, December 21, 2006:

It is a truism of American life that we’re too darn messy, or we think we are, and we feel really bad about it. Our desks and dining room tables are awash with paper; our closets are bursting with clothes and sports equipment and old files; our laundry areas boil; our basements and garages seethe. And so do our partners — or our parents, if we happen to be teenagers.

   But contrarian voices can be heard in the wilderness. An anti-anticlutter movement is afoot, one that says yes to mess and urges you to embrace your disorder. Studies are piling up that show that messy desks are the vivid signatures of people with creative, limber minds (who reap higher salaries than those with neat “office landscapes”) and that messy closet owners are probably better parents and nicer and cooler than their tidier counterparts. It’s a movement that confirms what you have known, deep down, all along: really neat people are not avatars of the good life; they are humorless and inflexible prigs, and have way too much time on their hands.
   Last week David H. Freedman, an amiable mess analyst (and science journalist), stood bemused in front of the heathery tweed collapsible storage boxes with clear panels ($29.99) at the Container Store in Natick, Mass., and suggested that the main thing most people’s closets are brimming with is unused organizing equipment. “This is another wonderful trend,” Mr. Freedman said dryly, referring to the clear panels. “We’re going to lose the ability to put clutter away. Inside your storage box, you’d better be organized.” Mr. Freedman is co-author, with Eric Abrahamson, of “A Perfect Mess: The Hidden Benefits of Disorder”…. read more

I have a big feature article in the new (November) issue of The Atlantic, which just came out. It’s the magazine’s annual look at “Brave Thinkers,” and it leads off with my longish profile of Dr. John Ioannidis, who has spent most of his career studying and exposing the many problems with published medical research. My book Wrong opens with Ioannidis, and comes back to him a few times. But in the article I go much deeper into his work, his background, his personality and his thinking–I spent several days with him in Greece for the article–and say more about what his discoveries mean for medicine and doctors’ ability to treat us effectively. It’s not a pretty picture. Think of the article as an extremely long, important and (I’d like to think) colorful post for this blog.

By the way, sorry for the long gap since the last post, I got caught in an extended perfect storm of article deadlines, and travel for research and speaking engagements. (And it’s not over yet.)

Meanwhile, on another semi-personal note that affects this blog, I’ve been transitioning to some new projects. I’ll say more about these projects in the coming months, but for now let me just say I have a new special interest in obesity and weight loss, which I (and many others) think have become the biggest single health threat to the length and quality of our lives that is also potentially fixable. The blog’s basic theme isn’t really changing, but a lot of the posts will be about issues relating to obesity and weight loss. Note that I’ve parenthetically added the new special interest in obesity to the blog’s title. I hope it goes without saying I’d love to get feedback from MSOMed readers on this partial change in focus.

A new study getting some very prominent and excited press, including a big feature spot on the Today show this morning, has found that drinking two glasses of water before a meal is the secret to losing weight. The Today show and the articles have noted that the study was a randomized controlled trial, which as we all know is the gold standard of medical research. So get ready to guzzle the H2O, and finally lose those pounds!

Or maybe not. Diet studies are terrible, and for a number of reasons. They’re of course subject to all the same problems that all medical studies suffer from: poor design, bad measurement, researcher and subject bias, patient selection problems, poor analysis, and more. Randomized controlled trials–in which, in the simplest case, subjects are randomly divided into two groups, one of which gets a special treatment and the other doesn’t–can help with some of these problems, because in theory whatever mistakes are made or biases take hold in the study with one group, the same should apply to the other group, so any differences in the results with the two groups should be the real deal. Well, it doesn’t always work out that way, even when an “RCT” is done right. But in this case, as with most diet studies that claim to be RCTs, it wasn’t done even close to right. The really big advantage to RCTs, as I wrote in my last post, comes from their being “blind,” meaning that both researchers and subjects (“double blind” is the technical term) don’t know which subjects are in which group. That’s what really has at least a shot at eliminating biases. But you can’t blind diet studies, because, for example, you can’t fool people into thinking they’ve just drunk two glasses of water when they haven’t, or vice-versa. And that means that the researchers and subjects are free to be very biased toward the treatment under study. They all want the research to pan out, and they tend to find ways to make sure it does. It’s also questionable whether non-blind randomized studies are truly randomized. The problem here is that once people in the study are subjected to the different treatments, you can start getting very different drop-out rates in the two groups, sometimes right up front. If you agreed to be in a study of some great new diet technique, and then were immediately told you’re going to be put on a plain old diet without getting to even try the technique, wouldn’t you be more tempted to just blow it off? After all, you’ve been on diets, they didn’t work–why bother trying again with the same old thing? The researchers are probably hoping you’ll fail on it so that their technique will prove effective, and they may communicate discouragement. It’s a mess of a study set-up.

It gets worse. Diet studies rarely go on long enough to determine whether people actually keep the weight off, which as we all know is the real question. People tend to lose weight on almost any kind of diet, typically over a period of a few months–but then they gain it all back, and often more, over the next year or so. This water-drinking study as originally published some months ago went on for 12 weeks, a classic set-up for misleadingly concluding that the diet was effective. But here’s where things get interesting. Though the original study only lasted 12 weeks, the newly released findings have the researchers claiming in public statements that the water-drinkers in the study on average kept the weight off for a year, and even lost a bit more. This is an extraordinary claim that sets off all sorts of alarm bells for me. Careful diet studies almost never report that sort of longer-term weight-loss success. And while the water-drinkers reportedly lost and kept off an average of 17 pounds after a year, the non-water-drinkers in the study lost and kept off an average of nine pounds after a year. Frankly, I’m more impressed by the non-water-drinking results–it’s even more unusual for the non-gimmicked dieters in a study to keep weight off that long. In looking at the original 12-week study, I didn’t see any specifics about what, if anything, the researchers did (or reasonably could have done, for that matter) to ensure that the subjects in the groups dieted in a way resembling the way people typically diet, that they didn’t receive some special encouragement or coaching, that the two groups were treated identically by researchers, that the groups accurately reported what they were eating and drinking, that the water-drinking group really drank the right amount of water before eating and that the non-water-drinking group didn’t drink a lot before or during meals. It also wasn’t clear what special pains, if any, were taken to make sure that bias wasn’t introduced by drop-outs, or by people who didn’t comply with the instructions or accurately report data. (It has been shown again and again that inaccurate self-reporting, which is a problem in many health studies, is particularly egregious in diet studies.) Whatever problems crept into the 12-week study, I can’t imagine it was anywhere but downhill in extending it to a year. The researchers certainly sound in the original report like they were trying to be careful, but there’s really only so much researchers can do, and the result in most diet studies is that there are potential holes big enough to drive a car through. And that certainly seems like the case with this study. I can’t help noting that the authors are releasing these newer findings before they have been peer-reviewed. Now, I’ve written at some length elsewhere that the peer review process isn’t nearly the guarantee of quality that it is often taken to be by the public and by journalists. But non-peer-reviewed data is generally considered one step up from fairy tales in the research community, and I wonder if the claims of this study might change after peer review.

But you don’t have to accept any of my reasoning about why this, or any other diet study, is likely to be fairly useless in telling you what’s going to help you or anyone else lose weight. Here’s a better reason to reject these studies: Scientific studies have found that anything is the secret to losing weight, including low carbs, high carbs, low fat, high fat, big breakfasts, small breakfasts, lots of veggies, lots of fruits, lots of nuts, snacking, not snacking, and on and on–and scientific studies have also found that every one of these gimmicks doesn’t work, including drinking a lot of water. There’s only one reasonable explanation for these conflicting findings: the scientific studies just aren’t very reliable. They fail to shine any light on the problem. We don’t have to agree on why that’s the case, but it’s hard to argue it isn’t the case.

It’s easy to explain why the water-drinking trick doesn’t work in the real world. It’s a type of diet I call a “sickening” diet. These diet gimmicks work by making you physically uncomfortable in some way, with the result that your appetite is dulled, as usually happens when you don’t feel great. Any diet that forces you to eat or drink a large amount of some type of not-especially-appealing food or beverage will have this effect. Some people don’t feel well even when eating low carb or low fat, which helps to explain why these diets may help many lose weight initially. The way the water throws your appetite off is obvious: it fills your stomach, and then when you add food on top of it you get an uncomfortably full sensation. None of these sickening diets are especially effective in the real world for what should be an obvious reason: people get sick of sickening themselves. They don’t like it. They don’t want to cram in another grapefruit, or celery stalk, or glass of olive oil, or chunk of meat. Go ahead, try the two glasses of water trick. Let me know how long you go before you find yourself staring at those two glasses of pre-dinner water and feeling, perhaps even with a touch of nausea, that the last thing you want to do is force them both down your gullet. Even worse, as the water or other sickening agent becomes more and more repulsive over time, your desire to go back to normal levels of eating (or normal varieties of food, if you’ve been restricting yourself in some way), will grow, especially as you lose weight, and you’ll become hungrier and will experience intense cravings. No wonder people gain it all back and more.

Folks, people have been trying to lose weight by drinking a lot more water for at least four decades. I remember my mother forcing down eight glasses a day in the 1970s under the Stillman diet. She didn’t keep the weight off, and neither did most people who have struggled with dieting and tried the water trick. If this silly gimmick worked, you’d have known it a long time ago.

Now having said that, let me say that drinking while dieting is a pretty good idea, as long as we’re talking about modest, non-repulsive amounts of water or any low-calorie beverage. It’s easy to mistake thirst for hunger, and it’s certainly true that having at least a little something in your stomach can temporarily help take the edge your appetite. So sure, step up your water- or tea- or diet-coke-drinking, within reason. Just about every diet expert has been saying that forever, and it very well might help your diet efforts a bit. I’m not surprised the water drinkers in this new study lost and kept off more weight than the non-drinkers. I’m just highly skeptical of the claim that it makes a big difference, and especially of the claim that it can make the difference between your dieting successfully or not over the long term. The researchers (and especially the mass-media reports) seem to believe that it will, but I don’t think that belief is anywhere close to being supported by the study. If you’re tempted to believe it, then I sincerely wish you the best of luck in getting yourself to stuff down two full glasses of water before sitting down to every meal you take in for the rest of your life, and in restricting how much you eat at all of these meals because of it.

The latest option for losing weight via surgically reducing stomach size: Having most of your stomach cut away and taken out by a procedure performed mostly through your mouth and down your throat. (A bit of the work is done through small slits in your abdomen.) What little is left of the stomach is then stitched up to form a mini-stomach, with the obvious result that it only takes a relatively small amount of food to fill it up. Other, more-established options for stomach surgery include the gastric bypass, a more extensive procedure in which in addition to the stomach being downsized it is also reconnected to a shorter span of intestine, cutting down on calorie absorption, and the lap band, in which much of your stomach is squeezed off by an adjustable band.

These procedures are often discussed as if they have been proven safe and effective in studies. But keep some points in mind when considering how “proven” a surgical technique is. There are (with a very few exceptions) no good, ethical ways to conduct randomized, blind trials of surgical procedures. It would be hard enough to do the randomization, in which patients are (to oversimplify a bit) randomly assigned to either a group that will receive the surgical procedure or a group that will not, because not many patients are willing to have surgical decisions made for them by the toss of a coin. But even if that issue were overcome, the real problem would be with blinding, which means patients don’t know which group they’re in. It’s not easy to get people to not know whether they’ve had surgery or not–and even if you could pull it off it’s even tougher to do it in a way that doesn’t run afoul of ethical considerations. (It’s been attempted, believe it or not, but hell was raised.) The result is that studies of newer surgical procedures are mostly observational studies, which means doctors just note how patients seem to be doing. (I’m not counting animal studies. Neither should you. No matter what researchers say, on average these studies don’t translate well to humans.) Some procedures that have been widely performed for many years get “case control” (epidemiological) studies, which look back at groups of patients who have received the treatment and compare them to groups who have not. While you can’t fully tell how trustworthy a study is just by noting what type of study it is–it’s wise to consider all medical studies at least potentially somewhat untrustworthy–as a rough generalization randomized, blind trials tend to be more trustworthy than case-control studies, and observational studies are the bottom of the barrel, though to be sure there are lots of exceptions and qualifications to this pecking order.

In short, the fact that gastric bypass, and, to a lesser extent, lap band procedures, have been found in case-control studies and many observational studies to be fairly safe and effective should not be taken very seriously. These types of studies can be subject to intense biases on both the parts of doctors and patients–they generally want the procedures to succeed, and may distort their observations and do a poor job of recording all relevant data. Researchers also typically don’t have access to all the data they need to fully analyze the situation. (For example, in the New England Journal of Medicine article I reference just above, the authors to their credit note that the study ignores the possibilities–likelihoods, I’d reckon, though the authors beg to differ–that people who had the surgery got much more or better medical attention afterward than other obese people get, and that they may have been healthier to begin with.) The procedures are a huge money-maker for surgeons, and the patients are often desperate to lose the weight and may be eager to convince themselves and others that they’ve done the right thing and that things are going to work out well. These biases, which can be and often are devastating to the reliability of medical studies, are exactly what randomized, blind studies attempt (if not always successfully) to eliminate. That’s why many surgical procedures are widely accepted as safe and effective, only to eventually prove themselves not so safe and/or less effective than other, less invasive and less risky treatments–the list includes many types of heart-related operations, back-pain-related operations, brain surgeries, and on and on.

This newest stomach-reduction procedure has been performed on one person so far. And yet surgeons just about everywhere will soon be able to perform it on anyone they deem suitable, providing nothing obviously terrible happens immediately to this one patient as a result. True, these sorts of “laparoscopic” (performed through small incisions and/or through the mouth, using a tiny camera and light to allow the surgeon to see what’s going on in there) procedures tend on average to be less risky than conventional, large-incision versions of the same surgery, but there can be reasons why they are sometimes less safe and/or less effective in some ways–for example, the restricted access and vision may result in a sloppier job of cutting or stitching, or in missing problems with bleeding.

And there are reasons to question whether the people who are getting these procedures really need them enough to justify the risks. (Though the twice-aforementioned NEJM study found that overall death rates went down for obese people who had gastric bypass surgery due to fewer deaths from heart disease, diabetes and cancer–findings that for a number of reasons I suspect may exaggerate the benefits of the procedure–around one percent died from the surgery itself shortly after the operation, and many had complications.) The people who sign up for these operations are typically obese people who have tried to lose the weight through diet and exercise but have failed, and whose health appears to be at some risk. But trying to compare the potential payoff from trying to diet and exercise to the likely benefits of surgery is a bit of a fixed contest. I’ve been studying the question of why people fail with diets and exercise, and it’s clear that the success rates would likely be much, much higher if overweight people weren’t getting such almost uniformly terrible advice about how to diet and exercise, and if their motivation to lose weight through dieting and exercise weren’t on average so low. Why is their motivation often so low? For one thing, they keep hearing from many researchers and obesity experts that they’ll probably fail with diet and exercise, and for another they’ve been led to believe that surgery provides a much easier and surer option that will provide all the same benefits of a successful diet and exercise program without any of the work–these procedures essentially are intensely marketed much the way cosmetic surgery is hawked, because they’re similarly ridiculously profitable. (Another issue is whether the success rate for dieting and exercising is really anywhere near as low as we’ve been told–but that’s a more complicated story.) Meanwhile, the success and benefits of surgical weight-loss keep getting pumped up through studies that are likely somewhat biased and otherwise flawed.

I don’t doubt there are obese people who, when the whole picture is soberly assessed, might be making a reasonable decision to undergo one of the procedures. But I suspect it’s a fraction of the number of people who are actually getting the surgeries. I was especially appalled to see a big and very well-researched and well-written cover story in the Atlantic earlier this year on the subject of obesity essentially advocate for, or at least seriously suggest the idea of, the government putting its anti-obesity money into handing out gastric bypass surgery for the obese, presumably instead of into research, education, good-nutrition programs, more-exercise initiatives and other anti-obesity public-health initiatives. The article, which for sure in many ways was terrific, more or less declared dieting and exercise to be a waste of time–a point of view which is pushed by many published medical studies relying on highly biased and flawed set-ups (as evidenced, for example, by the fact that findings from these studies widely and sometimes sharply conflict), but which flies in the face of the significant documented success rates that hundreds of real-world programs have achieved with obese people (and which has also been backed up by other published studies, for what it’s worth).

I think this sort of give-up-on-dieting-and-exercise-and-hold-out-for-surgery(-or-a-pill) thinking is dangerous, and is in fact a contributing factor to the obesity epidemic. If you’re thinking of getting some form of stomach-reduction surgery, I’d urge you to do some more research into the possible complications and into dieting and exercising options, and to discuss what you find with your doctor. And if your doctor is pushing you toward the surgery and especially if he seems biased toward it, try a second opinion from a doctor you have good reason to believe has an objective view about both diet-and-exercise programs and surgical alternatives. Many, many obese people have shed the weight and kept it off through dieting and exercise. Shouldn’t you be absolutely sure you’ve given dieting and exercise your very best shot under the right program before you have one or even two of your major organs hacked up and extensively rejiggered?

A new study claims it shows that the children of pregnant mothers who put on lots of weight are much more likely to be obese years later than the children of mothers who don’t put on as much weight in pregnancy. The announcement of the findings has triggered comments from other researchers and experts suggesting that this study is simply the latest in a series of studies supporting the notion that women who eat too much during pregnancy are placing their children at greater risk of obesity.

Nope. This new study tells us very little, and none of the others tells us much, either. The only sort of study that would have at least a small chance of giving us some real insight into the effect of eating during pregnancy on children’s lifelong tendency toward obesity would be one that randomly assigned many, many pregnant women to either a group that was somehow made to put on a relatively large amount of weight during the pregnancy, or a group that was somehow made to put on a relatively small amount of weight. Then the study would have to follow the weight gain of the children in the two groups for a few decades. Such a study would still be plagued with potential confounders, since, for example, it wouldn’t be blinded (the women and the researchers would likely know which group each woman was in), and the two groups of mothers might start behaving very differently from one another because of the particular way in which their weight had been controlled. But it doesn’t matter, because no one is likely to do that study–I can’t imagine researchers being able to line up enough pregnant women willing to participate, and even if they did that they’d be able to effectively control the two group’s weight gains during pregnancy, or even if they could that the whole project would make it through ethics review boards. So all researchers can do is track the weight gain of a bunch of pregnant women, wait to see what happens to their children, and then go back and look for links. This approach almost always leads to utterly unreliable results, because we have no way of knowing what causes what in the resulting observed links, and indeed the links themselves are often merely artifacts of bad data and/or bad analysis.

All these pregnant-weight-gain studies really show at best is that mothers who tend to put on too much weight are more likely to have children who put on too much weight. Not only is that painfully unsurprising, but it’s tremendously useless information. It doesn’t tell us whether that’s happening because of a genetic predisposition for weight loss that gets handed down from parent to child, or because mothers tend to pass healthy or unhealthy eating habits onto their children in some way, or because exposing very young children or possibly even fetuses to excessive intakes of food or to less-healthy foods somehow changes their metabolisms or alters their appetite mechanisms. (That first study’s conclusions seem to vaguely imply the latter, for what that’s worth, which I think is very little.) The studies give even fewer clues as to what ought to be done about the situation. If it’s genetic, there’s probably little point in addressing what pregnant mothers eat, which seems wrong. If the mother’s eating is programming the child in the womb in some way, then what the mother eats during pregnancy becomes critical–and what and how the child eats after birth is not so important, a conclusion that defies everyday observation and common sense. If the mother’s weight gain is “transferred” to the child via the food attitudes and habits that the child picks up from a mother who simply doesn’t do a good job of controlling her weight, then genes or what happens during pregnancy is largely irrelevant. I think there’s a pretty good chance that what we’re seeing here is some combination of the three–genes, changes that take place during pregnancy, and (probably especially) family eating habits passed on to the child later on. But the studies don’t help pin any of this down in the least.

So what should pregnant mothers take away from all this? Come on, you know the answer to that. They should eat sensibly (and as per doctor’s advice), avoid either excessive or insufficient weight gain during pregnancy (unless the doctor orders otherwise), and most important (in my opinion and in the opinion of most of the many experts I’ve interviewed on this subject) work hard to transmit generally accepted good eating habits to their children, starting from birth and continuing on through all of childhood. (If your doctor disagrees with that last bit, get a new doctor.) I doubt we’ll ever see a study that provides convincing evidence that doing anything else makes sense.

Update: A problem with the study that I describe in this post has surfaced within the scientific community, and has even received mass-media attention. It’s a technical problem of exactly the sort I warn about in the post. The only thing that’s unusual here is that the problem has been called out prominently.

A new study has shown that living into your nineties and beyond is a matter of having the right genes, according to recent headlines. The technique that the researchers employed to identify the genes of long life is much the same one behind many of the discoveries of links between genes and traits or genes and disorders that we hear about frequently. To oversimplify a bit, the game involves looking at many genes in a large group of people who have some trait in common, and then trying to spot which particular genes turn up more frequently in these folks than in a large group people who don’t have the trait.

I recommend not paying much attention to this study, or to most studies that purport to have identified key genes behind some important trait or disorder, and for a few reasons. First of all, this basic gene-linking technique, although an impressive one and getting better, is still beset with a variety of inherent limitations and potential flaws that often lead to simply misidentifying genes as being closely linked to a trait in most people when they are not, and misses many and possibly most of the genes that actually influence the trait. Part of the problem is that the technique is a highly statistical one based on probabilities involving all sorts of manipulations that can leave the picture convoluted and murky. In addition, the very notion that a relatively small number of genes is likely to play a huge role in determining a trait or disease risk is for the most part a mistaken one, or at best a gross oversimplification. Genes act not alone or even as a small band, but rather as a vast orchestra. Even more confusing, genes can be “turned off” so that they might as well not be there, or partly turned on so that they contribute only weakly to a trait or disease risk. What turns genes on and off? Each other, the environment, other molecules in our cells, and strips of DNA that aren’t even parts of genes. It gets worse: Genes can appear in single form, or in multiple copies that increase their influence in ways most gene tests don’t detect. They can pop out of their slot in the genome, replicate themselves, and then reinsert themselves somewhere else in the genome. They can flip around backwards, which also isn’t detected by gene tests, even though a backwards gene can behave differently. Throw in the fact that genes can be changed by viruses, and that any protein made by a particular gene can do different things in different people, or different things in the same person at different times depending on what else is going on in the body, and you’ve got complexity of such staggering breadth and depth that scientists who are being honest about the situation admit they’ll be lucky to make a small dent in the task of sorting it all out in their lifetimes.

Consider: A 2009 study of about 6,000 people came up with a technique for predicting the height of a person based on looking at the 54 genes most closely linked to height. The results turned out to be one-tenth as accurate as averaging the heights of both parents and adjusting for sex, a technique introduced in 1886 by statistician Sir Francis Galton. University College Dublin biomolecular researcher Helen Colhoun and colleagues have estimated that 95 percent of studies that find gene links are just plain wrong. Harvard genetics researcher Joel Hirschhorn and colleagues surveyed the medical literature on 166 published genetic links to illness that were each examined in at least two other published studies, and found that only six of the links–less than four percent–held up in all the studies. John Ioannidis of Tufts University Medical School has calculated the average odds of a gene-link study being right at one out of hundreds, or worse, depending on the disease. In the case of schizophrenia, for example, Ioannidis reckons a one out of 2,000 chance of a gene-link study being right. His estimates suggest you would have almost as good a chance of identifying some types of genuine gene links by throwing darts at a diagram of the relevant sections of the human genome as you would by reading research journals. Researchers claim their hit rate has improved quite a bit in the past few years, but we won’t know the extent to which that’s true for a while to come. Needless to say, all these substantial uncertainties and vagaries often get swept under the carpet in the reported findings of an exciting new study, and are almost always ignored in the mass-media reports.

None of this is to say that the study in question here didn’t correctly identify many genes that may in fact in some cases be potentially important contributors to living a very long life. But even if the claim is largely or at least partly right, it’s still a misleading one. When we hear it, we assume that we’ll need many or most of these genes to have a good shot at a long, healthy life–and that’s an absolutely unwarranted conclusion. People who live extraordinarily long lives need everything to go in their favor, including many things that may be largely irrelevant to those of us who hope to keep decent health “merely” into our 80s. Assuming we need the genes common in people who live to be 100 may be like assuming a good-performing car needs to have the components common to race cars.

Even if it does indeed help quite a bit to have the right genes in order to live a long and healthy life, it also helps to get a lot of other things right. What has come through from all the genetic studies more clearly than anything else is that for most of us genes really don’t influence your destiny as much as, or at least any more than, your behavior does. Folks, if you want to live relatively long lives, and perhaps more importantly stay relatively healthy late into that life, there is powerful evidence that eating sensibly, exercising, breathing clean air, avoiding undue stress, and refraining from obviously bad habits such as smoking, heavy drinking and not wearing seatbelts, can for many and possibly most of us trump genes. In fact, for all we know, some of the genes that the very-long-lived had in common in this study were genes that influence healthy behaviors–in which case anyone engaging in the behaviors would get the same benefits, whether they had the genes or not.

Your fate is probably not genetically sealed–it is much more likely that your health is to a large extent under your control. Don’t let announcements of marvelous gene-link breakthroughs lead you to abdicate that responsibility.

One recent set of headlines noted that an experimental, cutting-edge drug for the difficult-to-treat disease hepatitis C isn’t looking good for FDA approval because there’s evidence it can harm patients without much helping them. That’s terrible news for hepatitis C sufferers, of course, but most of us might just shrug, figuring that’s exactly the sort of outcome we might well expect from many and perhaps most unproven, high-tech treatments that are undergoing trials. What the public is much less aware of is that the same dubious benefit-to-risk situation applies to many of the drugs that are already widely prescribed. In just the past few days we’ve read headlines about how the diabetes drug Avandia, which has been taken by more than a million people, has now been strongly linked by big studies to potentially fatal heart disorders and strokes. And also making the news is a smaller study that suggests testosterone gel–increasingly commonly prescribed to older men to counter some of the debilitating effects of aging, including loss of bone density, muscle mass, mobility, energy and libido–may be strongly linked to heart disorders, at least in much older men with low mobility.

This isn’t just some small run of bad luck for the drug world. These sorts of problems are inherent to the very nature of drugs. Our bodies function via chemical “pathways” in which a vast number of proteins made by our genes interact with each other in ways so staggeringly complex that scientists struggle mightily to identify small pieces of the picture. When you take a drug, the hope is that it will go in and interrupt or boost some isolated part of a pathway that’s contributing whatever disorder you may be suffering from. But disorders can rarely be neatly pinned down to a simple pathway, which means there isn’t a clean target for the drug, which means the drug isn’t likely to hit the problem head on. What’s more, everyone has different genes, and even when we have the same relevant genes they may be turned “on” or “off” in different ways in different people, so that our pathways our different, which means that even when a drug is somewhat effective in one person it might not be so in another. Even worse, the drug is almost certainly affecting other pathways that are important to our health, with the effects rippling through the body in ways that may well cause harm. These basic problems are why we’ve seen a stream of much-hyped, widely used drugs yanked from the market after years, leaving behind a trail of people who weren’t much helped, or of people who suffered or even died from the drugs. Vioxx is probably the best-known example from the past several years, but recent headlines have listed many others.

Indeed, it would be miraculous if a drug could go in, completely solve a problem, and do nothing else–virtually no drugs have ever been found that pull that trick off. Instead, the vast majority of our drugs have much messier effects on us: they sort of help some people, hurt others, and don’t do much of anything for yet others. Even established, major drugs don’t work on 40% to 75% of people, according to a 2005 review paper in The New England Journal of Medicine, and the variation in effectiveness and risk of side-effects tends to be much greater for newer, less-proven drugs. The picture gets even worse when you’re talking about patients who are on multiple drugs, which can react with each other in any number of ways that are rarely well understood and often altogether unstudied. We don’t get all this from the findings of most studies, because they tend to report averages, and are often presented in a way that make it sound as if a drug can do a lot for you, with only some chance of causing you problems. That may be sort of true, but there’s really no way of getting around the fact that for most drugs you’re rolling the dice to some extent as to whether the drug is going to help or hurt, if it does anything at all. There’s hope genetic tests will eventually boost the odds further in your favor, but right now the claims you hear in the news that this is happening today is mostly hype, with just a few exceptions.

I want to make it clear I am in no way advocating pressuring your doctor to take you off any of your drugs–doctors admit in surveys to often succumbing to patient pressure on some drug decisions even if they don’t think the decision is medically best for the patient–and I most certainly don’t want anyone stopping drugs on their own. But there’s no reason why you shouldn’t ask your doctor to explain in more detail how likely a drug is to help, how likely it is to hurt, and why he thinks it’s worth it. By the same token, you might want to know what applicable drugs she’s decided aren’t right for you, and why. Why shouldn’t you be involved in a decision that essentially involves placing a bet on risks and benefits to your health? I always bow to my doctor’s judgment, but she’s willing to explain how she’s arrived at it, and she’s interested in getting my input when it’s a matter of weighing the odds, as is usually the case with medications.

If you want a wonderful illustration of the paucity of clear information with which many and possibly most health studies provide us, and in particular how hard it is to wring any sort of credible insight out of weight-loss studies, look no further than the study currently making headlines by claiming that it proved the effectiveness of school programs designed to reduce obesity.

The USA Today report on the study kept it really simple: “The health interventions significantly lowered body mass index (BMI) in children in the 85th percentile or more. Overweight or obese kids in intervention schools were 21% less likely of being obese by the end of 8th grade. The program also lowered average levels of fasting insulin and the number of students with a waist at or over the 90th percentile in intervention schools.”

But the HealthDay report provided some additional information: “….The program failed to reduce the overall numbers of overweight and obese schoolchildren–those numbers fell by 4 percent over three years whether the 42 middle schools in the study had such initiatives or not, the researchers report…. [and] children from both types of schools had the same average blood sugar levels and the same percentage of students with elevated blood sugar.”

OK, so let’s review: The interventions made a big difference with overweight kids–but didn’t make the kids much less overweight. And the interventions reduced the risk of diabetes–but didn’t prevent the high blood sugar levels that are the main indicator of diabetes or pre-diabetic conditions. Got that?

Here’s what apparently happened. The schools that had the interventions saw their initially overweight kids become on average very slightly less overweight over the course of the study–just enough of a drop, it seems, to push many of them from being just heavy enough to be classified as overweight to just missing being heavy enough to be classified as overweight. Voila! Not much reduction in weight (compared to height) for the overweight kids, but a significant drop in the number of them that were “officially” overweight. The interventions did little or nothing to keep kids who weren’t initially overweight from becoming overweight. And there was almost no difference between what happened in the schools that had the interventions and the schools that didn’t–in both schools, there was a very slight drop over the course of the study in the kids’ weight (compared to height).

As for the diabetes risk, the kids in the intervention schools saw slightly lower levels of insulin in their blood, which is generally associated with a lower diabetes risk. But it’s a less important indicator than blood sugar levels, which weren’t improved in the intervention schools. That’s a great example of how scientific findings get muddled and often outright derailed by the choice of what gets measured in a study. (It’s a problem I discuss at more length in an article in the July/August 2010 issue of Discover Magazine.)

Aside from the confusing and somewhat misleading way these results were presented in the press release, and made worse in some media accounts–it seems to me an honest presentation would have simply stated up front that the interventions didn’t seem to offer any clear evidence that the interventions worked–there were complicating issues in the way the study was conducted. For one thing, the intervention schools not only made changes to the food available to the kids, but it also required longer and more intense physical activity. Yes, most of us (including me) believe these two factors go hand in hand when it comes to reducing obesity. But it may be that one of these factors has a much bigger effect than the other, and it is probably the case that some schools will find it much easier to implement only one, and not the other. So studies need to try to tease these two apart. But the bigger problem, I think, is that the schools that didn’t get the official interventions were given money to make whatever changes they felt like making to their nutrition and fitness programs. Who knows what these schools did with the money? We don’t even know what we’re comparing here.

I certainly believe we have to come up with school-based approaches to reducing the astonishingly high child obesity rates, and I applaud any effort to study how that might be done. Unfortunately, this study’s results seem to join the long train of obesity study findings that cast little light on the subject. Presenting the findings in a way that makes it sound as if a marvelous breakthrough was achieved only makes things worse.

Newsflash: Exercise is good for your health. So is cutting down on high-fat meats, and on sugar and refined carbs such as white flour and white rice. Are you surprised? I’m not, personally, but it sounds as if a lot of people out there are finding these notions hard to believe, because we keep seeing health headlines that announce the latest study to discover that following these straightforward and generally well-accepted suggestions lowers the risks of various diseases, and of obesity.

Why do studies keep pounding these points home, when most people who actually care what studies have to say on these questions (as most people in America probably do not) have long since come to believe they’ve heard enough to make up their minds? Well, for one thing, that’s how science is supposed to work–you keep validating theories with new tests, because the new tests may be better than the old ones, and might turn up different answers or at least new twists on them. And the accumulated results from many studies give scientists a much better handle on what’s really going on. (The far bigger problem in science is that the vast majority of published research isn’t validated at all, but that’s a different story.)

Not only that, but the fact of the matter is that every so often an at least fairly credible study turns up evidence that suggests these standard healthy habits may not do as much for us as we’d like to think, and may even hurt. Thus there was the recent scientific review that suggested there is little solid evidence that any of the recommended lifestyle and nutritional habits lower the risk of Alzheimer’s, even though we’ve been hearing that they do for years. There are studies that, in violation of what virtually everyone who has lost weight and kept it off knows, have found that exercise doesn’t make any difference in controlling weight, a claim transmitted without doubt or qualification by former U.S. FDA commissioner David Kessler in his best-selling book on obesity. There are studies that find that eating a lot of saturated fat is just fine (these are pushed by the Atkins diet fanatics, of course), and others that conclude people would be thinner and healthier if they loaded up every morning on a high-carb breakfast. More than one study has found that being overweight may help you live longer.

So what’s up with all that? Probably several things. First of all, bear in mind that scientists themselves have recognized (as I’ve documented at length elsewhere) that even most prestigiously published medical research is refuted within a few years. Research is hard enough when done impeccably, but it is in fact plagued with surprising levels of sloppiness, bias, fudging and other distorting shenanigans. So no results should be surprising–someone, somewhere will manage to screw a study up and get exciting but false results that some perfectly good journal will be thrilled to publish. In addition, different researchers who are supposed to be studying the same thing often end up measuring very different things in very different ways, and in very different sorts of people. Thus one study might conclude that a certain diet lowers the risk of, say, diabetes while another concludes it does not–simply because one of the studies measured blood-sugar levels in healthy patients while the other measured deaths from diabetes-related complications in very ill patients. (These measurement mismatches may be part of the problem with the studies that find obesity can be protective–it seems to prove so only for older patients and those with extensive heart disease, perhaps because old or sick people who are thin face greater risks from nutritional shortages than from the slower-acting ravages of obesity, or so it has been speculated.)

But there’s an even more basic reason why we shouldn’t be surprised that studies conflict on the healthfulness of nutrition and habits. There are thousands of factors that come into play when it comes to the ways various nutrients, habits and other environmental factors affect us, and they interact in a complex web that science may not yet have barely begun to unravel. Scientists try to tease the roles of individual factors out in their studies, but rarely can they honestly claim to have succeeded. (And let’s not even get started here on the fact that they often base their claims on animal studies.)

All of these challenges conspire to distort study results and further confuse an already confusing picture. At least with exercise, satured fat and refined carbs there’s a reasonably strong consensus of evidence, if not an unblemished one. That’s why I’m going to keep exercising, and limiting saturated fat and refined carbs. I’m not fully confident any of that will do much for me, but at least I know I’m playing the odds.

Blood-pressure pills appear to be linked to higher risks of cancer, according to a big review study making headlines. Should these results be at least a little worrisome to you if you’re on one of these medications? In my opinion, yes–it’s worth discussing with your doctor. (Please don’t stop taking your pills on your own.) This study has many of the earmarks of more trustworthy research. It’s a meta-study, which means it looked at the results from several previous studies, so that if some of the studies were troubled there’s still some hope the results from the better studies will partly compensate for those problems. The studies were randomized, controlled trials, which means the people in them were randomly assigned to either take the medication in question or not take it, which can eliminate a lot of the problems studies run into. (I’m oversimplifying, but that’s the basic idea.) And the studies were large, totaling some 60,000 patients, so there shouldn’t be as big a chance for statistical flukes as there would be in small studies. None of these characteristics guarantee reliable findings, but they can all help.

It’s worth noting that some articles describing this new study report that the study found that people on the drugs have a one percent higher risk of getting cancer. But other articles say the findings indicate risks go up on average about 10 percent, and for lung cancer a whopping 25 percent. The seeming contradiction here is due to the media’s ongoing, near-complete and deeply problematic failure to specify and make clear the difference between absolute risk and relative risk. When your absolute risk of contracting a disease goes up by, say, ten percent, it means that, for example, if your risk of the disease was 14 percent, it is now 24 percent (14 plus 10), a big difference. When your relative risk goes up by ten percent, then if it was 14 percent, it is now around 15 percent (14 plus ten percent of 14)–not such a big difference. So in this example, an increase in relative risk of about 10 percent translates to an increase in absolute risk of a mere 1 percent. Research institutions typically want the public to be impressed with the importance of their results, so they often emphasize to the press relative risk, which is a generally misleadingly high number. But in this case–perhaps because there is concern that people on blood-pressure pills will overreact and stop taking them–there seems to be more interest than usual in reporting the change in absolute risk, which tends to be a very low number. Assuming you don’t carry any special risk factors for lung cancer (such as smoking), you only have a tiny chance of getting it, and if you add on the apparent increased risk from blood-pressure pills, you now have a slightly less-tiny chance of getting lung cancer. Still, your chances of getting cancer of any type aren’t so tiny, and a one percent increased absolute risk is nothing to sneer at. A percentage here, a percentage there, and pretty soon you’re taking years off your life.

If you read about the study, you might note that even though the study found an increased risk for blood-pressure-pill takers of getting cancer, it didn’t find an increased risk of them dying from it. Unfortunately, you shouldn’t take this as reassuring. Most cancer studies fail to convincingly link increased risk factors to mortality, usually simply because it tends to take people many years to die of cancer (if they do die from it, that is), and studies usually don’t go on long enough to follow people to their graves. (That’s another reason, by the way, that researchers love running studies on mice instead of people–they’re conveniently short-lived. We humans can be annoyingly enduring, from a medical-research point of view.)

Talk to your doctor about it. It may be that if you were just on the borderline of high blood pressure but have special risks for cancer, she may want you off the pills. But if your doctor feels your risks from uncontrolled high blood pressure outweigh the possible slightly increased risks of cancer turned up by this study, I’d trust him. In the unlikely chance your doctor doesn’t know what you’re talking about when you ask about this study, I myself would tell him that I’d be happy to sit there and wait while he looked the study up.

Two recently released European reports have raised the possibility that scientists who helped fuel world fear of an H1N1 flu crisis were motivated by ties to drug companies enriched by the ensuing rush to stockpile medications. The reports, which were from credible parties, including a Council of Europe committee and the British Medical Journal, noted that several of the World Health Organization scientists behind that agency’s pressuring of governments to take dramatic action in response to the flu had received consulting or other fees from the drug companies, but that WHO hadn’t bothered to identify these potential conflicts of interest. (Though the scientists in question had previously identified these ties in research-journal papers). With H1N1 turning out to fall far short of the devastating pandemic WHO and others predicted, many of the world’s governments have ended up with vast unused stores of antiviral and other medications snatched up at WHO’s urging.

There’s no evidence that the scientists ginned up the warnings in order to channel sales to drug companies that had been good to them, and it seems unlikely that’s the case. Scientists widely agreed there was plenty of reason to fear that H1N1 could become widespread and deadly. And we surely want our scientists, institutions and governments to err on the side of caution in these matters. It may even be that the rapid action taken by governments at WHO scientists’ prodding was at least partly why H1N1 appears to have been something of a dud, so that the wasted drugs are in a way a sign of WHO’s competence rather than its failure. (WHO actually continues to warn there’s still a real danger of H1N1 getting out of hand.)

Still, science is supposed to be tough on biases, and especially blatant conflicts of interest. It has long been standard to insist that scientists openly and clearly state any ties they have to any corporations when there is any chance that their work could be seen as benefiting those corporations. But there’s little question that conflict of interest remains a big problem in medical research. A 2003 Journal of the American Medical Association review of conflict-of-interest meta-studies involving some 67 conflict-of-interest studies and 398 other research reports confirmed a strong correlation between industry sponsorship and positive findings. (I provide sources in Wrong for that last and the following points).  And the problem may be worse than it looks, because companies often disguise that they are behind certain findings by paying university researchers to put their names to studies actually conducted and written up by the companies–so-called “ghost authorship,” a problem that infects about three-quarters of industry-backed drug trials, one study found According to another study, at least 30 percent of published medical researchers have been in a potential conflict-of-interest situation, and as few as 2 percent of researchers fully disclosed that fact. Every single one of the 170 psychiatric experts contributing to the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM IV) used to diagnose psychiatric disorders had financial ties to manufacturers of psychiatric drugs.

Remember this situation every time you hear of research that concludes a drug works well, or a new type of test is effective, or, yes, an infectious disease is about to wreak havoc on the world unless governments buy up all the treatments they can get their hands on. That’s not to say the specific research is likely to be wrong–but it’s another reason why we need to be wary consumers of research conclusions.

Breast cancer vaccine: As cancer researchers themselves like to point out privately, they’ve cured cancer thousands of times–in mice. It’s this simple: mice aren’t people. Treatments that work on them usually don’t work on people, or don’t work as well, or carry serious side effects. This non-translation seems particularly problematic with cancer, given that mice appear to develop cancer a lot more readily than humans do (or at least researchers have come up with effective techniques for seeing to it that they develop it, typically through genetic engineering, as was the case with this vaccine research), and the cancers they get appear to be a great deal more treatable than the human counterparts. There’s no obvious reason to believe this particular development will fare any better than its predecessors if and when it moves into human trials.

Antidepressants increase miscarriage risk: The research on the efficacy and safety of antidepressants is all over the place. The bulk of solid-seeming research seems to suggest they don’t do much more for most people than a placebo does, but there are all sorts of difficulties involved in trying to pin down improvements in the states of depressed people, and the picture is further blurred by the clearly demonstrated corruption of the literature due to drug-industry influence. The safety picture for antidepressants is even murkier, because, for one thing, depressed people may be more vulnerable to a range of health problems whether or not they’re on medications. This particular study doesn’t seem very strong, based as it is on looking back at the medication history of women who had miscarriages–an approach that can be wide open to all sorts of statistically distorting effects. Still, it’s well-recognized that fetuses can be highly vulnerable to side effects in medications, and most people would agree that it’s wise to err on the side of caution when it comes to giving meds to pregnant women., so it would be wrong to simply dismiss this study. Hopefully better studies will clarify things.

Sunscreen can cause cancer: It’s hard to know what to make of the ongoing flap about the possibly carcinogenic effects of sunscreen; the research is weak. The specific research behind the recent headlines that suggest the vitamin A in many sunscreens is carcinogenic was carried out on mice, and doesn’t seem to be getting much respect from most of the medical community. The headlines are motivated by press releases from the Environmental Working Group, a health-advocacy group that appears to have a strong anti-chemical, anti-industry bias. (Probably the safer bias to have, if you have to have a bias, but still, bias is bias.) I can’t help noting, though, that at least one other study has found that slathering skin creams on mice followed by exposure to tanning-type light seems to induce cancer. The evidence doesn’t seem very compelling, but on the other hand there’s nothing far-fetched about the notion that chemicals designed to penetrate and alter the skin in some way increase vulnerability to forms of sunlight known to cause cancer. Common sense suggests that staying out of the sun, or blocking it with opaque or reflective materials or ointments (like zinc oxide), is probably a better way to avoid skin cancer. Having said that, I have to point out that exposure to sunlight is linked to lower risk of almost all other forms of cancer, most likely because it raises natural production of vitamin D. It’s possible that taking vitamin D supplements can provide the same protection, allowing skipping the sun exposure, but studies testing that notion have come up with conflicting findings. Hey, glad I was able to clear this whole sunscreen business up.

The doctor who claimed to have proven a link between autism and routine childhood vaccinations has been stripped of his license to practice medicine, the latest in a long string of humiliations. It turned out that his now-discredited study was carried out while he was quietly being well-paid by lawyers who were suing vaccine makers, and was based on blood taken from kids at a birthday party who were each slipped eight bucks for their trouble.

Let’s review: A corrupt charlatan tried to foist a sloppy and possibly fraudulent piece of research onto a vulnerable community–many parents of autistic children are haunted by the lack of good explanation for the disorder–but the persistent work of mainstream scientists laid bare the scheme. And that’s what we expect from science, isn’t it? There may be the very occasional peddlers of junk science who sneak into reputable journals, but science surely if slowly brings their egregious transgressions to light, and expunges them and their work like so much gristle.

But here’s an alternate view to consider: Even prestigious medical journals routinely run dubious studies that are tainted by bias, corruption or incompetence, and are rarely called out on it–but because this particular study got so much attention the scientific community for once actually went through the trouble of exposing it.

That probably seems far-fetched, but it’s much closer to the truth. The evidence supporting this point of view is so plentiful and consistent that it’s daunting to contemplate having to narrow it down to a mere handful of quick points so as not to make this an absurdly long post, but let me try, drawing on my research in Wrong (where these factoids are elaborated on and sourced): About two-thirds of the findings published in top medical journals turn out to be wrong or significantly exaggerated. Nearly all studies that find a particular type of food or a vitamin lowers the risk of disease fail to hold up. In spite of all the hype for gene discoveries that promise to deliver insights into our personal disease risks and customized treatment, studies have found that less than 10 percent of the published studies that identify gene links to disease hold up. Studies suggest that only about one percent of the scientific studies based on fraudulent data are identified and reported. On average roughly one-third of medical researchers surveyed in various studies admit to having committed or become personally aware of at least one act of research misconduct within the previous three years. Two-thirds of the drug-study findings that indicate a drug may cause harm are not fully reported by researchers. A third of the studies published in top medical journals contain statistical errors. And here, at least in this context, is the kicker: An estimated ninety-five percent of medical findings are never retested.

Ironically, the occasion of the rare public outing of one out of an ocean of bad studies actually serves to make medicine look good, because the story comes off as the community successfully rooting out the tiny percentage of bad work that sneaks in.  Rarely does anyone openly point out in public that bad studies are the rule rather than the exception, let alone actually identify many of the vast number of studies that are not trustworthy, let further alone actually correct the record with better research.

Yes, in this case the system worked, because the findings were so inflammatory in terms of the politics of the large autism community, and because they so blatantly contradicted what scientists were already sure they knew about the subject. But it’s a real exception to how the system fares, not a general validation. And that’s the insight we really ought to take away from this sorry episode.

I should probably kick off this blog with some general comments about its motivation and goals. I’ve spent the last three years working on a book about why experts usually end up misleading us. The book, called WRONG, is just coming out now. (You can learn more about it at www.freedman.com.) Although WRONG looks at all sorts of experts and their highly flawed advice, I think the problem that has the most impact on the most people is the way that published medical research runs off the rails. We’re fairly bombarded on a daily basis with the latest medical findings, typically ripped from the pages of prestigious medical journals, and echoed in our most respected mass media publications and shows, telling us which foods and habits will lower our disease risk, what new pills show promise, which genes predict our health fates, and so on. There’s just one little problem: According to scientists themselves, most of these published findings will turn out to be wrong or highly exaggerated. Eating those foods probably won’t help you much, the pills probably won’t work well, the genes probably won’t tell you much. And this turns out to be true not just of what you read in the newspapers or see on TV or on news websites, but also of the work published in our top medical journals. (Journalists tend to make things worse, but we don’t cause the basic problem.) There are many reasons why most published findings turn out to be flawed, which I explore in detail in WRONG, and will be touching on as the need arises in this blog. But for now, let me just state that there is a wealth of consistent and mostly uncontested evidence from highly respected scientists that the published medical research you hear about is on average not trustworthy. (And if like many people you’ve been led to believe that large, randomized controlled trials can generally be trusted, you’ve been misled.)

You generally won’t hear about problems with the credibility of medical research when you’re alerted to new findings in the mass media, because almost no one has an incentive to focus on the problems or tell you about them. Why would most researchers take the trouble in their papers or in their interviews to highlight why their work is probably wrong? Why would most journalists take up space in their articles and time in their on-air reports to discuss why you probably shouldn’t take the reports very seriously? You might think you could count on your doctor to figure out which published findings really ought to be put into practice, but you probably can’t–most doctors don’t have the training, time or inclination to deconstruct the studies they read about in journals in order to spot the potential problems with them.

I’m not a doctor, I’m not a scientist, I’m not an academic researcher. I’m a journalist. I’ve spent a lot of time speaking to, and reading the work of, a lot of highly regarded scientists who study the problems with medical research, and I think I now know a thing or two about these issues. I’d like to share the insights I’ve gained (and the insights of the many scientists I’ll be interviewing for this blog), and help people apply them to the current news about medical findings, so that we can all make better guesses about which of these findings are really likely to improve our health and possibly save our lives, and which of them ought best be regarded with great skepticism. (Sadly, most of them will fall into the latter category.) You, of course, will use your judgment to decide how much you should trust me and this blog and the people I quote in it, as we all should with all the findings and advice and pronouncements that stream at us from so many sources.

I want to make it very clear that I’m not interested in turning anyone away from science and mainstream medicine–far from it. I’m as appalled as anyone by junk science, and I’m not an advocate of alternative medicine (though I respect its placebo effect, as do many mainstream clinicians). I trust my doctor, and believe many if not most scientists to be among society’s heroes, if highly imperfect ones. I believe that science is exactly the right way to get at truth. But we all need to understand just how messy and biased and flawed a process science is, how slowly and rarely it actually manages to close in on the truth, and how it becomes even further distorted when it’s communicated to us through journals and the media.