Freakonomics author Steven Levitt: Incompetent? Unethical?

Steven Levitt (along with Donohue), the well known economist author (along with Dubner) of Freakonomics, advocated the theory that the advent of legalized abortion in the 1970s is responsible for much of the steep and persistent drop in the crime rate during the 1990s. The research behind this claim was a paper by Donohue and Levitt. However, as Foote and Goetz [2005] showed, it turned out to be wrong.

Donohue’s and Levitt’s mistake is so elementary as to suggest incompetence or the intent to mislead. These two possibilities also are consistent with Donohue’s and Levitt’s criticism of John Lott’s research showing that allowing honest citizens to carry concealed weapons reduces violent crime.

Donohue has written copiously on Lott’s work, all of it negatively. However, as Plassman and Whitney show in their Stanford Law Review article “Confirming More Guns Less Crime”, Ayres and Donohue’s work appears to be substandard, including statistical errors and claims not supported by the data. Moreover, Donohue has continued to make negative claims in the face of several papers pointing out his errors. Here, too, the mistakes and continued apparently incorrect claims are so elementary and pervasive as to suggest incompetence or the intent to mislead.

Levitt has taken up Donohue’s cudgel and attacked Lott in Freakonomics, saying:

Then there was the troubling allegation that Lott actually invented some of the survey data that supports his more-guns/less-crime theory. Regardless of whether the data were faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate his results, they found that right-to-carry laws simply don’t bring down crime.

In fact, Lott’s results have been replicated and supported by other researchers. This was known in the field. Indeed one researcher sent the following email to Levitt.

I also found the following citations – have not read any of them yet, but it appears they all replicate Lott’s research.

Levitt’s response was:

It was not a peer refereed edition of the Journal. For $15,000 he was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.

Levitt’s response misstated the facts.

It seems that Levitt now has a record on at least two important issues, abortions and guns. In both cases he has confidently and repeatedly espoused a position that is probably wrong and that he should have known was probably wrong. In one case, he has compounded the error by apparently acting unethically, i.e., by possibly libeling a competent researcher in lieu of admitting his own likely mistake.

Here is Foote and Goetz’s abstract.

State‐level data are often used in the empirical research of both macroeconomists and microeconomists. Using data that follows states over time allows economists to hold constant a host of potentially confounding factors that might contaminate an assignment of cause and effect. A good example is a fascinating paper by Donohue and Levitt (2001, henceforth DL), which purports to show that hypothetical individuals resulting from aborted fetuses, had they been born and developed into youths, would have been more likely to commit crimes than youths resulting from fetuses carried to term. We revisit that paper, showing that the actual implementation of DL’s statistical test in their paper differed from what was described. (Specifically, controls for state‐year effects were left out of their regression model. ) We show that when DL’s key test is run as described and augmented with state‐level population data, evidence for higher per capita criminal propensities among the youths who would have developed, had they not been aborted as fetuses, vanishes. Two lessons for empirical researchers are, first, that controls may impact results in ways that are hard to predict, and second, that these controls are probably not powerful enough to compensate for the omission of a key variable in the regression model.

Here is the abstract from Ayres’s and Donohue’s paper “Shooting Down the More Guns, Less Crime Hypothesis”.

John Lott and David Mustard have used regression analysis to argue forcefully that “shall issue” laws (which give citizens an unimpeded right to secure permits for concealed weapons) reduce violent crime. This article shows that the claim has support from certain facially plausible statistical models, but that these are rejected by a variety of statistical tests. Estimating more statistically preferred disaggregated models on more complete data, we show that in most states shall issue laws have been associated with more crime. Using our expanded data set and our preferred jurisdiction-specific regression model, we show that more states have experienced an upturn in crime than have experienced a downturn in crime after enacting the law and that the apparent stimulus to crime tends to be especially strong for those states that adopted in the last decade. We estimate that on net the passage of the law in 24 jurisdictions has increased the annual cost of crime somewhere on the order of half a billion dollars. We also provide an illustration of how our jurisdiction-specific regression model has the capacity to generate more nuanced assessments concerning which states might profit from a particular legal intervention.

Here is an excerpt from the introduction to Plassman’s and Whitney’s Stanford Law Review article “Confirming More Guns Less Crime”.

Quite a few empirical papers have examined the impact of right-to-carry laws on crime rates. Most studies have found significant benefits, with some finding reductions in murder rates twice as large as the original research. Even the critics did not provide evidence that such laws have increased violent crime, accidental gun deaths, or suicides.

Unlike previous authors, Ian Ayres and John Donohue claim to have found significant evidence that right-to-carry laws increased crime. However, they have misread their own results. The most detailed results they report. following the change in crime rates on a year-by-year basis before and after right-to-carry laws were adopted.clearly show large drops in violent crime that occur immediately after the laws were adopted. Their hybrid results showing a small increase in crime immediately after passage are not statistically significant and are an artifact of fitting a straight line to a curved one. When one examines a longer period.from 1977 to 2000.even this type of result disappears.

Ayres and Donohue.s efforts have been valuable in forcing others to reexamine the evidence, extend the dataset over more years, and think of new ways to test hypotheses, and we appreciate their efforts. They are both highly regarded and well-known for their research, such as claiming that the legalization of abortion can account for half the drop in murder during the 1990s. Unfortunately, their research on this issue inaccurately describes the literature and also fails to address previous critiques of their work. For example, Ayres and Donohue claim that "[w]hen we added five years of county data and seven years of state data, allowing us to test an additional fourteen jurisdictions that adopted shall-issue laws, the previous Lott and Mustard findings proved not to be robust". All their tables report results for "Lott's Time Period (1977-1992)" and compare those estimates with the "Entire Period (1977-1997)". Yet, whatever differences in results arise, they are not due to the inclusion of more data for a longer period. Their paper gives a misleading impression as to how much their research extends the data period, since Lott's book and other work examined both the county and state data up through 1996. Ayres and Donohue's work thus extends the county-level data by one year, from twenty to twenty-one years.