If you want a wonderful illustration of the paucity of clear information with which many and possibly most health studies provide us, and in particular how hard it is to wring any sort of credible insight out of weight-loss studies, look no further than the study currently making headlines by claiming that it proved the effectiveness of school programs designed to reduce obesity.
The USA Today report on the study kept it really simple: “The health interventions significantly lowered body mass index (BMI) in children in the 85th percentile or more. Overweight or obese kids in intervention schools were 21% less likely of being obese by the end of 8th grade. The program also lowered average levels of fasting insulin and the number of students with a waist at or over the 90th percentile in intervention schools.”
But the HealthDay report provided some additional information: “….The program failed to reduce the overall numbers of overweight and obese schoolchildren–those numbers fell by 4 percent over three years whether the 42 middle schools in the study had such initiatives or not, the researchers report…. [and] children from both types of schools had the same average blood sugar levels and the same percentage of students with elevated blood sugar.”
OK, so let’s review: The interventions made a big difference with overweight kids–but didn’t make the kids much less overweight. And the interventions reduced the risk of diabetes–but didn’t prevent the high blood sugar levels that are the main indicator of diabetes or pre-diabetic conditions. Got that?
Here’s what apparently happened. The schools that had the interventions saw their initially overweight kids become on average very slightly less overweight over the course of the study–just enough of a drop, it seems, to push many of them from being just heavy enough to be classified as overweight to just missing being heavy enough to be classified as overweight. Voila! Not much reduction in weight (compared to height) for the overweight kids, but a significant drop in the number of them that were “officially” overweight. The interventions did little or nothing to keep kids who weren’t initially overweight from becoming overweight. And there was almost no difference between what happened in the schools that had the interventions and the schools that didn’t–in both schools, there was a very slight drop over the course of the study in the kids’ weight (compared to height).
As for the diabetes risk, the kids in the intervention schools saw slightly lower levels of insulin in their blood, which is generally associated with a lower diabetes risk. But it’s a less important indicator than blood sugar levels, which weren’t improved in the intervention schools. That’s a great example of how scientific findings get muddled and often outright derailed by the choice of what gets measured in a study. (It’s a problem I discuss at more length in an article in the July/August 2010 issue of Discover Magazine.)
Aside from the confusing and somewhat misleading way these results were presented in the press release, and made worse in some media accounts–it seems to me an honest presentation would have simply stated up front that the interventions didn’t seem to offer any clear evidence that the interventions worked–there were complicating issues in the way the study was conducted. For one thing, the intervention schools not only made changes to the food available to the kids, but it also required longer and more intense physical activity. Yes, most of us (including me) believe these two factors go hand in hand when it comes to reducing obesity. But it may be that one of these factors has a much bigger effect than the other, and it is probably the case that some schools will find it much easier to implement only one, and not the other. So studies need to try to tease these two apart. But the bigger problem, I think, is that the schools that didn’t get the official interventions were given money to make whatever changes they felt like making to their nutrition and fitness programs. Who knows what these schools did with the money? We don’t even know what we’re comparing here.
I certainly believe we have to come up with school-based approaches to reducing the astonishingly high child obesity rates, and I applaud any effort to study how that might be done. Unfortunately, this study’s results seem to join the long train of obesity study findings that cast little light on the subject. Presenting the findings in a way that makes it sound as if a marvelous breakthrough was achieved only makes things worse.
David H. Freedman 