Earlier this week I wrote about some common pitfalls in statistical analysis. (See Statistical Games #1)
This morning’s Arizona Daily Star presented a good example of a questionable statistical study:
The study claims that “Women who said they walked briskly had a 37 percent lower risk of stroke than those who didn’t walk.” “The research involved about 39,000 female health workers 45 or older enrolled in the Women’s Health Study. The women were periodically asked about their physical activity. During 12 years of follow-up, 579 had strokes.”
The apparent problem with this study is that the “37% lower risk” is a relative risk which says nothing about your actual chances of getting a stroke. Also, a red flag is “women who said.” This was not a clinical trial, but relied on the memory and veracity of the study participants. The basic input data were completely uncontrolled.
Since 579 women of the 39,000 had strokes, this yields a stroke incidence 1,484 per 100,000 for the study period. (The incidence per 100,000 is a common way to report incidence of a condition in a population.)
I searched the internet to find data on incidence of stroke in the general population and met with limited success. Here are two studies I found:
“A 1998 study by researchers at the University of Cincinnati Medical Center suggests that the number of strokes in the United States may be dramatically higher than previously estimated. According to the study, approximately 700,000 strokes occur in the United States every year.” This works out to an incidence of 233 per 100,000 people in the U.S.
Another study, published on BioMedCentral: “There were 712,000 occurrences of stroke with hospitalization and an estimated 71,000 occurrences of stroke without hospitalization. This totaled 783,000 occurrences of stroke in 1996, compared to 750,000 in 1995. The overall rate for occurrence of total stroke (first-ever and recurrent) was 269 per 100,000 population (age- and sex-adjusted to 1996 US population).” At least this study had some clinical evidence and the results of the two studies are in the same ball park. Apparently none of the studies considered confounding factors such as diet, smoking, or genetics.
So I ask, if the incidence of stroke in the general population is somewhere near 250 per 100,000, and the “brisk walkers” reported a stroke incidence of 1,484 per 100,000, where is the evidence that walking lowers the incidence of stroke? One could just as easily conclude from the study that women who walk briskly have a 6 times higher chance of suffering a stroke.
By the way, did anybody catch the switch I pulled? I said that the women had a stroke incidence of 1,484 per 100,000 for the study period. But the study period was 12 years. The other studies I quoted referred to annual rates. If the women’s study incidence of strokes occurred evenly throughout the 12 year period then there would have been 48 per year or 123 per 100,000, or about half that of the general population. Maybe walking does help.
But then again, the walkers had only a 37% lower incidence of stroke not 50%. The report of the study didn’t say how many were walkers. That would be important in evaluating the study.
Aren’t statistics fun? The point of this essay is that one must be wary of statistical studies, especially those that do not have controlled clinical or experimental data, and make sure they are not comparing apples to oranges.