The gas stove furor and the flawed study that started it

Blair King discusses why the study that created the anti-gas crowd is fatally flawed. There is no evidence that gas stoves should be banned for health reasons.

BK is a Professional Chemist with an Interdisciplinary PhD in Chemistry and Environmental Studies with an undergraduate degree in Chemistry and Biology.

Once again, those driven by agenda and power as opposed to what is good for the Country, are using selective flawed statistics to mislead the rest of us. About the only thing you can count on from this ilk is dishonesty.

Here is BK's perspective.

---------------------------------------------

The news has been full recently with stories about the risk of childhood asthma caused by natural gas stoves. As someone who specializes in risk assessment and has experience with indoor air chemistry this seemed like it was right up my alley. As I went digging through the research; however, I discovered that the research seemed less about providing a good scientific examination of the topic and and more about generating a lot of headlines and press discussion of the topic.

The furor is all derived from a recent study published in an open-source journal called Population Attributable Fraction of Gas Stoves and Childhood Asthma in the United States (Gruenwald et al., 2022). The paper itself doesn’t present any new data but rather applies a rather arcane type of mathematical attribution analysis (Population Attributable Fraction or PAF) to the results from a ten-year-old meta-analysis that summarized work from the 80’s and 90’s. Needless to say, the paper absolutely doesn’t advance the science in any useful manner and appears designed instead to induce political change rather than inform policy.

Two of the authors of the paper are Talor Gruenwald and Brady A. Seals. Many of us are familiar with these names as they both work for the Rocky Mountain Institute. For those not familiar RMI is:

an independent, non-partisan, nonprofit organization of experts across disciplines working to accelerate the clean energy transition and improve lives.

Now I’m not going to slag the RMI as it really does do good work. But it is absolutely fair to note that two authors who work for an organization that is dedicated to transforming the global energy system to secure a clean, prosperous, zero-carbon future for all might not be the totally objective scientists you want doing your research on natural gas stoves.

Before we get too deep into evaluating the data used in the paper, I think it is pretty important we start with a little background on the critical statistical tool used in this paper (PAF). As described in the literature PAF

is an epidemiologic measure widely used to assess the public health impact of exposures in populations. PAF is defined as the fraction of all cases of a particular disease or other adverse condition in a population that is attributable to a specific exposure.

That sounds like a pretty useful measure but there is a hitch. PAF has been around since the 1950s but a Google Scholar search of the term finds less than 17,000 hits. From an academic perspective, this tells you a lot about the technique. A statistical tool in epidemiology (a field that publishes thousands of papers a year) that has been around for 70 years and only appears in a few thousand papers must have some issues, and PAF absolutely does. The big complaint is that PAF doesn’t work when there are multiple confounding variables. The challenge for academics unfamiliar with the tool PAF is

found in many widely used epidemiology texts, but often with no warning about invalidness when confounding exists.

So let’s consider Asthma as a disease. According to the American Lung Association Asthma can be caused by: Family History (genetics), Allergies. Viral respiratory infections in youth, Occupational exposures. Smoking, Air pollution and Obesity. Do you know what a statistician would call each of those SEVEN different causes of asthma? Confounding variables! So here we have a statistical analysis that is invalid when used in the presence of confounding variables and we have a disease that can be caused by a half dozen other factors, that are not controlled for in the analysis.

Reading the Gruenwald et al paper carefully, one discovers the terms “confounding” and “variable” do not appear. It is thus possible the authors simply did not recognize the issues with this statistical tool for this type of analysis as that omission would typically result in a bench rejection in most well-respected journals.

Another challenge with this paper is the data used to derive its conclusions. The research for this paper started with an evaluation of the academic literature. The authors started where most authors on this topic start. With the 2013 Meta-analysis of the effects of indoor nitrogen dioxide and gas cooking on asthma and wheeze in children by Lin, Brunekreef and Gehring. This is a seminal paper on this topic and I have seen it cited numerous times by those opposed to fossil fuel stoves. The major problem with the paper is that it is old. While it was written in 2013, it relies almost entirely on research articles from the 1980’s and 1990’s. From the perspective of indoor air assessment that is like the Stone Age. A look at the supplementary material for the work shows that most of the studies included were, by modern perspective, very small and had little statistical power.

Given that knowledge the authors of Gruenwald et al., looked for newer work and but unfortunately found no new data. Why? Because

Full manuscripts (n = 27) were independently reviewed…none reported new associations between gas stove use and childhood asthma specifically in North America or Europe.

So there were 27 major studies they could have included in their analysis but the authors deliberately limited their inputs by requiring the work be done entirely in North America and Europe because they were looking for “similarities in housing characteristics and gas-stove usage patterns”.

By making this editorial choice the authors managed to exclude the definitive research on the topic: Cooking fuels and prevalence of asthma: a global analysis of phase three of the International Study of Asthma and Allergies in Childhood (ISAAC). The ISAAC study was

a unique worldwide epidemiological research program established in 1991 to investigate asthma, rhinitis and eczema in children due to considerable concern that these conditions were increasing in western and developing countries. ISAAC became the largest worldwide collaborative research project ever undertaken, involving more than 100 countries and nearly 2 million children and its aim to develop environmental measures and disease monitoring in order to form the basis for future interventions to reduce the burden of allergic and non-allergic diseases, especially in children in developing countries

The ISAAC study collected data from 512,7070 students between 1999 and 2004. It has incredible statistical power due to its massive sample size and one of its signature conclusions was:

we detected no evidence of an association between the use of gas as a cooking fuel and either asthma symptoms or asthma diagnosis.

Arguably, in any study to evaluate the “Population Attributable Fraction of Gas Stoves and Childhood Asthma in the United States” a massive, recent, international study that showed that there was no evidence of an association between natural gas as a cooking fuel and asthma might be considered relevant. But no, that landmark study was ignored in this analysis.

Even worse…and I can’t believe I am saying this, even the seminal meta-analysis by Lin, Brunekreef and Gehring barely met their standard. Of the 41 papers evaluated in that meta-analysis the Gruenwald et al authors chose only to consider 10 (those where all subjects were from Europe or the US). The limitation of relying solely on European and US data was nominally due to the “similarities” between housing characteristics in the US and Europe but it further degraded the statistical power of their analysis

Now I am not speaking out of school when I point out that houses in the US are really not more comparable to European homes than homes in Australia or Japan. Anyone who has ever travelled to Europe can attest to how similar their housing design is to US building and frankly American houses are not all that comparable either. I would argue that the differences between houses in Nevada and New Hampshire would greatly exceed the differences between those in Nevada and Australia. Thus, it is fair to ask whether imposing this restriction was really about maintaining internal consistency of the data or whether other factors might have played a role?

To conclude, I can only restate that the Gruenwald et al paper seems to have some clear challenges that would typically preclude it from consideration in a policy-making process.

• Its underlying data is of low statistical power.

• Its conclusion is directly contradicted by more recent studies with significantly greater statistical power. and

• It relies on a statistical tool that is considered invalid in situations with confounding variables yet it is being used to analyze an association that is absolutely rife with confounding variables.

Put simply, this is not the study I would rely on to make a major policy change that will affect millions of people and will cost billions to implement. As to its conclusion: are 12.7% of childhood asthma cases in the US attributable to cooking with natural gas? Based on the points above, that conclusion is almost certainly not the case.