The amazing fact about the Alarmist Insect paper is how wildly inadequate was the authors' methodology and data and the disconnect between what the authors' methodology and data implied and the authors' conclusions.
Sánchez-Bayoa and Wyckhuys (2019) set out to review and systematically assess “the changes in species richness (biodiversity) and population abundance though time” and “the likely drivers of the losses” of insects across the globe. The authors searched the online Web of Science database using the keywords [insect*] AND [declin*] AND [survey]. By including the word [declin*], there is a bias towards literature that reports declines, and the bias is not resolved by the procedure in which “additional papers were obtained from the literature references”. If you search for declines, you find declines. Searching for declines would have been appropriate, had the authors only aimed for evaluating the drivers of the declines. In the same vein, the statement “almost half of the species are rapidly declining” is unsubstantiated, as there are no data about the speed of the decline. Furthermore, the data are not extensive geographically (as the authors acknowledge) or taxonomically, so the conclusion that the current proportion of insect species in decline would be 41%, or that insects as a whole would be going extinct, are also unsubstantiated.
Our second criticism concerns the mismatch between the study objectives and the actual studies included. The authors state “Reports that focused on individual species...were excluded” and “We selected surveys that… were surveyed intensively over periods longer than 10 years”. Why, then, did they include a single-species study on Formica aquilonia which was conducted over four years only (see Sorvari and Hakkarainen 2007)? We did not scrutinize all the reviewed studies but just happened to be familiar with this one. Because Sánchez-Bayoa and Wyckhuys (2019) lumped together single species studies and continent-wide data sets, as well as primary field studies, various reports and expert opinions like the national IUCN Red Lists, analyses and interpretations were challenging. In fact, many of the “extinctions” in the reviewed papers apparently represent losses of species from individual sites or regions, and it is not straightforward to extrapolate to the extinction of species at larger spatial scales (see also Thomas et al. 2019). The extrapolation is also challenging because the study included only cases with detected declines.
Our third criticism concerns the misuse of the IUCN Red List categories (citation for IUCN 2009 is actually missing from the references) to assess extinction risk. At least in one case (McGuinness 2007), Sánchez-Bayoa and Wyckhuys (2019) lumped together species in the category ‘Data Deficient’ and ‘Vulnerable’. Because by definition there are no data for Data Deficient species to assess neither the decline nor the range size or population abundance, this means that the authors themselves designated a 30% decline (Vulnerable indicates > 30% decline) for Data Deficient species. This is not trivial, since 24% of the Vulnerable species were actually Data Deficient in McGuinness (2007). The use of the IUCN criteria is also poorly described. Did the authors solely use the number of threatened species as presented in the original articles, or did they also themselves designate declining species to different IUCN categories (not all countries follow the IUCN system)? And if the latter, did they consider the fact that the IUCN criteria assumes the decline has happened in ten years or three generations, whichever is the longer.
Putting the unsubstantiated claims about the extent of insect declines aside, there may also be a methodological complication regarding the drivers, because of the chosen indicator. The authors base their inference about the importance of the driver on the number or share of the papers where the driver is reported to have caused the declines. Number of reports is not a reliable indicator of the importance of the driver as it can simply reflect the interest of scientists or ease of studying certain drivers. More reliable conclusions about the importance of different drivers would have required reviewing also the drivers in studies without declines. Vote counting as conducted here, provides only limited, if any, information about the strength of the driver, which would be of interest for the conservation managers. Ideally, a formal meta-analysis with effect sizes of different drivers, and an unbiased sample of population trend studies including positive, negative and no effect would have provided a more complete picture of the declines and their relative strengths.
The final problematic issue with the paper is its strong language. Like noted by The Guardian, the conclusions of the paper were set out in unusually forceful terms for a peer-reviewed scientific paper. The text is rich in non-scientific intensifiers such as dramatic, compelling, extensive, shocking, drastic, dreadful, devastating, and others. This language is clearly reflected by the media with direct quotes, and with what media often does, by adding on to the already intensifier rich text. Exaggerated news made by the media itself are bad as they are, but similar exaggerations in the original scientific papers should not be acceptable. The current case has already seen corrections and withdrawals in the print media as well as in social media, and the first academic responses have been published (e.g. Thomas et al. 2019). As actively popularizing conservation scientists, we are concerned that such development is eroding the importance of the biodiversity crisis, making the work of conservationists harder, and undermining the credibility of conservation science.
--------------------------------------------
“Unless we change our ways of producing food, insects as a whole will go down the path of extinction in a few decades”. This is a verbatim conclusion of the recent paper by Sánchez-Bayoa and Wyckhuys (2019): Worldwide decline of the entomofauna: A review of its drivers. There is also another slightly less sweeping but still bold conclusion: “Our work reveals dramatic rates of decline that may lead to the extinction of 40% of the world’s insect species over the next few decades”. In an interview by Damian Carrington of The Guardian, the authors explained that they are not alarmist, but that they really wanted to wake people up. If measured by the global media attention, they succeeded. A version of their conclusions hit the headlines across the planet in mainstream media such as BBC News, Al-Jazeera, ABC News and USA Today. Unfortunately, even if not intentional, the conclusions of Sánchez-Bayoa and Wyckhuys (2019) became alarmist by bad design: due to methodological flaws, their conclusions are unsubstantiated.Sánchez-Bayoa and Wyckhuys (2019) set out to review and systematically assess “the changes in species richness (biodiversity) and population abundance though time” and “the likely drivers of the losses” of insects across the globe. The authors searched the online Web of Science database using the keywords [insect*] AND [declin*] AND [survey]. By including the word [declin*], there is a bias towards literature that reports declines, and the bias is not resolved by the procedure in which “additional papers were obtained from the literature references”. If you search for declines, you find declines. Searching for declines would have been appropriate, had the authors only aimed for evaluating the drivers of the declines. In the same vein, the statement “almost half of the species are rapidly declining” is unsubstantiated, as there are no data about the speed of the decline. Furthermore, the data are not extensive geographically (as the authors acknowledge) or taxonomically, so the conclusion that the current proportion of insect species in decline would be 41%, or that insects as a whole would be going extinct, are also unsubstantiated.
Our second criticism concerns the mismatch between the study objectives and the actual studies included. The authors state “Reports that focused on individual species...were excluded” and “We selected surveys that… were surveyed intensively over periods longer than 10 years”. Why, then, did they include a single-species study on Formica aquilonia which was conducted over four years only (see Sorvari and Hakkarainen 2007)? We did not scrutinize all the reviewed studies but just happened to be familiar with this one. Because Sánchez-Bayoa and Wyckhuys (2019) lumped together single species studies and continent-wide data sets, as well as primary field studies, various reports and expert opinions like the national IUCN Red Lists, analyses and interpretations were challenging. In fact, many of the “extinctions” in the reviewed papers apparently represent losses of species from individual sites or regions, and it is not straightforward to extrapolate to the extinction of species at larger spatial scales (see also Thomas et al. 2019). The extrapolation is also challenging because the study included only cases with detected declines.
Our third criticism concerns the misuse of the IUCN Red List categories (citation for IUCN 2009 is actually missing from the references) to assess extinction risk. At least in one case (McGuinness 2007), Sánchez-Bayoa and Wyckhuys (2019) lumped together species in the category ‘Data Deficient’ and ‘Vulnerable’. Because by definition there are no data for Data Deficient species to assess neither the decline nor the range size or population abundance, this means that the authors themselves designated a 30% decline (Vulnerable indicates > 30% decline) for Data Deficient species. This is not trivial, since 24% of the Vulnerable species were actually Data Deficient in McGuinness (2007). The use of the IUCN criteria is also poorly described. Did the authors solely use the number of threatened species as presented in the original articles, or did they also themselves designate declining species to different IUCN categories (not all countries follow the IUCN system)? And if the latter, did they consider the fact that the IUCN criteria assumes the decline has happened in ten years or three generations, whichever is the longer.
Putting the unsubstantiated claims about the extent of insect declines aside, there may also be a methodological complication regarding the drivers, because of the chosen indicator. The authors base their inference about the importance of the driver on the number or share of the papers where the driver is reported to have caused the declines. Number of reports is not a reliable indicator of the importance of the driver as it can simply reflect the interest of scientists or ease of studying certain drivers. More reliable conclusions about the importance of different drivers would have required reviewing also the drivers in studies without declines. Vote counting as conducted here, provides only limited, if any, information about the strength of the driver, which would be of interest for the conservation managers. Ideally, a formal meta-analysis with effect sizes of different drivers, and an unbiased sample of population trend studies including positive, negative and no effect would have provided a more complete picture of the declines and their relative strengths.
The final problematic issue with the paper is its strong language. Like noted by The Guardian, the conclusions of the paper were set out in unusually forceful terms for a peer-reviewed scientific paper. The text is rich in non-scientific intensifiers such as dramatic, compelling, extensive, shocking, drastic, dreadful, devastating, and others. This language is clearly reflected by the media with direct quotes, and with what media often does, by adding on to the already intensifier rich text. Exaggerated news made by the media itself are bad as they are, but similar exaggerations in the original scientific papers should not be acceptable. The current case has already seen corrections and withdrawals in the print media as well as in social media, and the first academic responses have been published (e.g. Thomas et al. 2019). As actively popularizing conservation scientists, we are concerned that such development is eroding the importance of the biodiversity crisis, making the work of conservationists harder, and undermining the credibility of conservation science.
No comments:
Post a Comment