A new study getting some very prominent and excited press, including a big feature spot on the Today show this morning, has found that drinking two glasses of water before a meal is the secret to losing weight. The Today show and the articles have noted that the study was a randomized controlled trial, which as we all know is the gold standard of medical research. So get ready to guzzle the H2O, and finally lose those pounds!
Or maybe not. Diet studies are terrible, and for a number of reasons. They’re of course subject to all the same problems that all medical studies suffer from: poor design, bad measurement, researcher and subject bias, patient selection problems, poor analysis, and more. Randomized controlled trials–in which, in the simplest case, subjects are randomly divided into two groups, one of which gets a special treatment and the other doesn’t–can help with some of these problems, because in theory whatever mistakes are made or biases take hold in the study with one group, the same should apply to the other group, so any differences in the results with the two groups should be the real deal. Well, it doesn’t always work out that way, even when an “RCT” is done right. But in this case, as with most diet studies that claim to be RCTs, it wasn’t done even close to right. The really big advantage to RCTs, as I wrote in my last post, comes from their being “blind,” meaning that both researchers and subjects (“double blind” is the technical term) don’t know which subjects are in which group. That’s what really has at least a shot at eliminating biases. But you can’t blind diet studies, because, for example, you can’t fool people into thinking they’ve just drunk two glasses of water when they haven’t, or vice-versa. And that means that the researchers and subjects are free to be very biased toward the treatment under study. They all want the research to pan out, and they tend to find ways to make sure it does. It’s also questionable whether non-blind randomized studies are truly randomized. The problem here is that once people in the study are subjected to the different treatments, you can start getting very different drop-out rates in the two groups, sometimes right up front. If you agreed to be in a study of some great new diet technique, and then were immediately told you’re going to be put on a plain old diet without getting to even try the technique, wouldn’t you be more tempted to just blow it off? After all, you’ve been on diets, they didn’t work–why bother trying again with the same old thing? The researchers are probably hoping you’ll fail on it so that their technique will prove effective, and they may communicate discouragement. It’s a mess of a study set-up.
It gets worse. Diet studies rarely go on long enough to determine whether people actually keep the weight off, which as we all know is the real question. People tend to lose weight on almost any kind of diet, typically over a period of a few months–but then they gain it all back, and often more, over the next year or so. This water-drinking study as originally published some months ago went on for 12 weeks, a classic set-up for misleadingly concluding that the diet was effective. But here’s where things get interesting. Though the original study only lasted 12 weeks, the newly released findings have the researchers claiming in public statements that the water-drinkers in the study on average kept the weight off for a year, and even lost a bit more. This is an extraordinary claim that sets off all sorts of alarm bells for me. Careful diet studies almost never report that sort of longer-term weight-loss success. And while the water-drinkers reportedly lost and kept off an average of 17 pounds after a year, the non-water-drinkers in the study lost and kept off an average of nine pounds after a year. Frankly, I’m more impressed by the non-water-drinking results–it’s even more unusual for the non-gimmicked dieters in a study to keep weight off that long. In looking at the original 12-week study, I didn’t see any specifics about what, if anything, the researchers did (or reasonably could have done, for that matter) to ensure that the subjects in the groups dieted in a way resembling the way people typically diet, that they didn’t receive some special encouragement or coaching, that the two groups were treated identically by researchers, that the groups accurately reported what they were eating and drinking, that the water-drinking group really drank the right amount of water before eating and that the non-water-drinking group didn’t drink a lot before or during meals. It also wasn’t clear what special pains, if any, were taken to make sure that bias wasn’t introduced by drop-outs, or by people who didn’t comply with the instructions or accurately report data. (It has been shown again and again that inaccurate self-reporting, which is a problem in many health studies, is particularly egregious in diet studies.) Whatever problems crept into the 12-week study, I can’t imagine it was anywhere but downhill in extending it to a year. The researchers certainly sound in the original report like they were trying to be careful, but there’s really only so much researchers can do, and the result in most diet studies is that there are potential holes big enough to drive a car through. And that certainly seems like the case with this study. I can’t help noting that the authors are releasing these newer findings before they have been peer-reviewed. Now, I’ve written at some length elsewhere that the peer review process isn’t nearly the guarantee of quality that it is often taken to be by the public and by journalists. But non-peer-reviewed data is generally considered one step up from fairy tales in the research community, and I wonder if the claims of this study might change after peer review.
But you don’t have to accept any of my reasoning about why this, or any other diet study, is likely to be fairly useless in telling you what’s going to help you or anyone else lose weight. Here’s a better reason to reject these studies: Scientific studies have found that anything is the secret to losing weight, including low carbs, high carbs, low fat, high fat, big breakfasts, small breakfasts, lots of veggies, lots of fruits, lots of nuts, snacking, not snacking, and on and on–and scientific studies have also found that every one of these gimmicks doesn’t work, including drinking a lot of water. There’s only one reasonable explanation for these conflicting findings: the scientific studies just aren’t very reliable. They fail to shine any light on the problem. We don’t have to agree on why that’s the case, but it’s hard to argue it isn’t the case.
It’s easy to explain why the water-drinking trick doesn’t work in the real world. It’s a type of diet I call a “sickening” diet. These diet gimmicks work by making you physically uncomfortable in some way, with the result that your appetite is dulled, as usually happens when you don’t feel great. Any diet that forces you to eat or drink a large amount of some type of not-especially-appealing food or beverage will have this effect. Some people don’t feel well even when eating low carb or low fat, which helps to explain why these diets may help many lose weight initially. The way the water throws your appetite off is obvious: it fills your stomach, and then when you add food on top of it you get an uncomfortably full sensation. None of these sickening diets are especially effective in the real world for what should be an obvious reason: people get sick of sickening themselves. They don’t like it. They don’t want to cram in another grapefruit, or celery stalk, or glass of olive oil, or chunk of meat. Go ahead, try the two glasses of water trick. Let me know how long you go before you find yourself staring at those two glasses of pre-dinner water and feeling, perhaps even with a touch of nausea, that the last thing you want to do is force them both down your gullet. Even worse, as the water or other sickening agent becomes more and more repulsive over time, your desire to go back to normal levels of eating (or normal varieties of food, if you’ve been restricting yourself in some way), will grow, especially as you lose weight, and you’ll become hungrier and will experience intense cravings. No wonder people gain it all back and more.
Folks, people have been trying to lose weight by drinking a lot more water for at least four decades. I remember my mother forcing down eight glasses a day in the 1970s under the Stillman diet. She didn’t keep the weight off, and neither did most people who have struggled with dieting and tried the water trick. If this silly gimmick worked, you’d have known it a long time ago.
Now having said that, let me say that drinking while dieting is a pretty good idea, as long as we’re talking about modest, non-repulsive amounts of water or any low-calorie beverage. It’s easy to mistake thirst for hunger, and it’s certainly true that having at least a little something in your stomach can temporarily help take the edge your appetite. So sure, step up your water- or tea- or diet-coke-drinking, within reason. Just about every diet expert has been saying that forever, and it very well might help your diet efforts a bit. I’m not surprised the water drinkers in this new study lost and kept off more weight than the non-drinkers. I’m just highly skeptical of the claim that it makes a big difference, and especially of the claim that it can make the difference between your dieting successfully or not over the long term. The researchers (and especially the mass-media reports) seem to believe that it will, but I don’t think that belief is anywhere close to being supported by the study. If you’re tempted to believe it, then I sincerely wish you the best of luck in getting yourself to stuff down two full glasses of water before sitting down to every meal you take in for the rest of your life, and in restricting how much you eat at all of these meals because of it.
David H. Freedman 
Hello David,I read your article in Discover magazine with great interest. As a consumer of research, I find that people who are not informed regarding how to read research, including journalists who publish the findings, that often research findings are exaggerated or just plain wrong. (pardon the pun) I am looking forward to reading your book. I guess what I am most interested in knowing is so how do we distinguish between what is good research and what is not? I like to be informed. Thanks.
Hi Louisa, you're really asking the key question, how can we tell the flawed scientific findings from the ones that will hold up? I wish there were a simple, sure-fire answer. I go into some detail on it in the book, of course, but let just me just mention a few of my rough rules of thumb: Be at least a little wary of all scientific findings; be especially wary of recent breakthrough findings, links between health risks and nutrition or lifestyle, and findings based on one or two studies, small studies or animal studies; and pay closer attention to findings supported by a consensus of researchers that has been building for some years. I also find that if you keep looking at studies critically, you really start to develop a sense of when a finding deserves some trust and when it doesn't. Of course, that sense, and all these "rules," often turn out to be wrong, too! Like I said, no easy answers here. The real problem is that the world is complex, and we need to be comfortable with the idea that most of what our experts come up with to explain it will be wrong. (By the way, you can get to the Discover article to which Louisa refers from my home page, http://www.freedman.com.)
Thank you David! Very helpful. Louisa
Surely the advantage of an RCT is random allocation to intervention not blinding. It is possible to blind in any controlled group.
You're right, of course, that blinding isn't a definitive characteristic of RCTs, and I shouldn't have implied it is, thanks for pointing that out. But it's often taken to be–when we hear about an RCT, we typically assume it's double-blinded, because if blinding is feasible then it's usually done, and it's one of the reasons RCTs tend to be more trustworthy than other types of studies. We don't often ponder the question of whether unblinded RCTs are necessarily all that much more reliable than blinded or even unblinded non-randomized controlled trials. Whether randomization is in fact more important than blinding to the trustworthiness of results would depend quite a bit on the study. If confounders are the big risk, then randomization is critical. If bias on the part of the researchers or subjects is the bigger risk, then blinding might well be far more important. When it comes to diet studies, randomization would certainly be helpful in limiting the problems with the many possible confounders (e.g. socio-economic factors are strongly correlated with obesity), but in my opinion blinding would often be even more helpful, because researchers are frequently highly invested in proving their theory right, and could easily treat the two or more groups differently in ways that distort the results. There could also be a big placebo effect operating unequally on the treatment groups. In either case, the fact that subjects were randomly assigned to the groups would be zero protection against resulting distortions in the findings.