John Lott shows why a widely quoted study by Adam Lankford comparing mass public shootings in the US to elsewhere was botched.
Given the history cited in Lott's paper it is difficult to conclude that Lankford errors too grevious and too frequent to be due to other than dishonesty.
Here is the link to Lott's paper.
Flawed data and statistics are a frequent occurrence in what you see from the media and gun control groups - and even from many academics. Lott is an exception.
Here are some excerpts.
-------------------------------------------
Executive SummaryA paper on mass public shootings by Adam Lankford (2016) has received massive national and international media attention, getting coverage in the New York Times, the Wall Street Journal, plus hundreds of other news outlets spanning at least 35 different countries. Lankford’s claim was that over the 47 years from 1966 to 2012, an enormous amount of the world’s mass public shooters -- 31% -- occurred in the United States. Lankford attributed this to America’s gun ownership.
Lankford claims to have “complete” data on such shooters in 171 countries. However, because he has neither identified the cases nor their location nor even a complete description on how he put the cases together, it is impossible to replicate his findings.
It is particularly important that Lankford share his data because of the extreme difficulty in finding mass shooting cases in remote parts of the world going back to 1966. Lack of media coverage could easily lead to under-counting of foreign mass shootings, which would falsely lead to the conclusion that the U.S. has such a large share.
Lankford’s study reported that from 1966 to 2012, there were 90 public mass shooters in the United States and 202 in the rest of world. We find that Lankford’s data represent a gross undercount of foreign attacks. Our list contains 1,448 attacks and at least 3,081 shooters outside the United States over just the last 15 years of the period that Lankford examined. We find at least fifteen times more mass public shooters than Lankford in less than a third the number of years.
Coding these events sometimes involves subjectivity. But even when we use coding choices that are most charitable to Lankford, his 31 percent estimate of the US’s share of world mass public shooters is cut by over 95 percent. By our count, the US makes up less than 1.43% of the mass public shooters, 2.11% of their murders, and 2.88% of their attacks. All these are much less than the US’s 4.6% share of the world population. Attacks in the US are not only less frequent than other countries, they are also much less deadly on average.
Given the massive U.S. and international media attention Lankford’s work has received, his unwillingness to provide even the most basic information to other researchers raises real concerns about Lankford’s motives.
“I say this every time we’ve got one of these mass shootings: This just doesn’t happen in other countries.” –Obama, news conference at COP21 climate conference in Paris, Dec. 1, 2015
“The one thing we do know is that we have a pattern now of mass shootings in this country that has no parallel anywhere else in the world.” –President Obama, interview that aired on CBS Evening News, Dec. 2, 2015
“You don’t see murder on this kind of scale, with this kind of frequency, in any other advanced nation on Earth.” – President Obama, speech at U.S. Conference of Mayors, June 19, 2015
Introduction
To justify the claims in these and other similar quotes, President Obama’s administration cited a then unpublished paper by criminologist Adam Lankford. The study received coverage in hundreds of news stories, with international news coverage in at least 35 different countries. Purporting to cover all mass public shootings around the world from 1966 to 2012, Lankford claimed that the United States had 31% of public mass shooters despite having less than 5% of the world population.
Major media outlets gave Lankford's claims uncritical coverage. Headlines accepted his findings as fact.
—The Wall Street Journal: “U.S. Leads World in Mass Shootings.”
—The Wall Street Journal (subheading): “U.S. produces more mass shootings than other countries.”
—The Los Angeles Times: “Why the U.S. is No. 1—in mass shootings.”
—Time magazine: “Why the US has 31% of the World’s Mass Shootings.”
— Newsweek magazine: “Study Sees Mass Shootings as ‘Exceptionally American Problem’.”
—Washington Post: “American exceptionalism and the ‘exceptionally American’ problem of mass shootings.”
—CNN: “Why the U.S. has the most mass shootings.”
— Sunday Morning Herald (Australia): “Why the U.S. is No. 1 in Mass Shootings.”
Similar coverage was given that year by USA Today, PBS Newshour, NPR, ABC Evening News, Fox News, and many hundreds of other outlets. The stories were carried by various wire services and covered by media outlets across the country. Many have covered the claim repeatedly. Coverage wasn’t just limited to the US or even the English-speaking world. It has received extensive attention in countries such as Australia, Austria, Argentina, Armenia, Brazil, Canada, China, Colombia, Costa Rica, Denmark, Egypt, Finland, France, Germany, Hungary, India, Indonesia, Iran, Ireland, Japan, Malaysia, Mexico, Peru, Portugal, Russia, Slovenia, South Africa, Spain, Sweden, Turkey, UK, Venezuela, Vietnam, and Cuba.
The research keeps being cited. In the last nine months, the New York Times has twice republished the same diagrams which show that the United States stands alone in its number of guns and mass public shootings. Since October 2017, USA Today has re-cited this research on three more occasions. Just this year, it has received coverage on television and twice in the Washington Post as well as Chicago Tribune, Houston Chronicle, Politifact, and Psychology Today. The Washington Post referred to it this year as Lankford’s “famous study.”
Since then, the media has continually used Lankford’s numbers and asserted that the US has an incredibly high rate of mass public shootings. Often, this is attributed to America’s high gun ownership rate.
Unfortunately, Lankford has not published or released his list of mass public shootings or even the number of cases by country or by year. He has even refused repeated requests to provide a list of the news sources or languages he used to compile his list of cases. This prevents anyone from doing even a rough check of his data. Only a couple news stories interviewed any researcher who might be critical of Lankford’s claims.
In his original paper, Lankford gave information on the number of cases for only four countries: France, Philippines, Russia, and Yemen. Just for the New York Times, Lankford provided information on the number of attacks by country, which was used to publish a series of graphs. But the observations in the Times’ graphs were labeled for only those same four countries.
For less developed parts of the world such as Africa or Latin America, it can be very difficult to obtain news stories from even a decade or so ago. It is downright impossible to obtain news stories on all of the cases of four or more people being killed in the 1960s or 1970s. So instead of looking at all 47 years that Lankford claims to study, we examined the last 15 years of his period of study: 1998 to 2012.
Lankford claims to have found 292 mass public shooters over the 47 years from 1966 to 2012, with 90 occurring in the United States and 202 in the rest of world. Over just the last 15 years of that period, we find 1,491 cases — 43 in the United States.
In the following discussion we show how sensitive the results are to decisions on what to include in the count. But even the most generous assumptions produce results show that mass public shooters, shootings, and murders from these attacks are very rare in the US compared to the rest of the world.
Conclusion
We tried to duplicate Lankford’s results. Coding these events sometimes involves subjectivity. But even when we use coding choices that are most charitable to Lankford, his 31 percent estimate of the US’s share of world mass public shooters is cut by over 95 percent. The US makes up less than 1.43% of the mass public shooters, 2.11% of their murders, and 2.88% of their attacks. These results show that the U.S. clearly has fewer mass public shootings and murders from these attacks than the average rate for the rest of the world.
Lankford is clearly wrong that the NYPD dataset “may be nearly comprehensive in its coverage of recent decades.” Over the 15 years that we studied, the NYPD dataset had 16 foreign attacks, with 27 killers and 393 killed. By contrast, we find 1,448 attacks, with 3,081 killers and 15,095 killed. The NYPD is very far from being comprehensive.. Lankford shouldn’t have ignored the NYPD warning that they had a “strong sampling bias against international incidents.”
Whenever possible, academics have a responsibility to make their data available so that other researchers can confirm their findings. This obligation is particularly important after the research has been published or received media attention or been used multiple times by the president. Lankford's refusal to provide either academics or the media with a list of his mass public shootings or to explain how he identified his cases should have raised real concerns among journalists who covered his paper. For many places, such as Africa, it is challenging to obtain cases from the last decade, let alone during the 1960s, 1970s, 1980s, and 1990s.
The massive difference in the number of cases that we have discovered and what Lankford claimed points to either extreme sloppiness or possible fraud. His refusal to share his data, his paper, and even methodology details, suggest that he may have known that his study contained dramatic flaws.
After compiling this data for the 15 years from 1998 through 2012, the last fifteen years studied by Lankford, it is clear that he missed an enormous number of cases. We have found about fifteen times more shooters in 15 years than Lankford claimed to find in 47 years. But however one counts these cases, the United States is well below the average country regarding either the frequency or murder rate from these attacks or their deadliness.
This data not only has implication for how the United States compares to other countries but also to previous claims about what might be responsible for these attacks. For example, Lankford’s claim that higher rates of gun ownership are associated with more mass public shooters completely disappears when this more complete data on mass public shooters is used (Lott, 2018).
No comments:
Post a Comment