Sunday, July 16, 2006

The Miami Herald's bad logic, bad statistics, and possible contribution to a higher death rate

The 7/16/06 Miami Herald has an editorial on air cargo safety. It follows “A Miami Herald investigative series last week -- Deadly Express by Ronnie Greene” that “described the air-cargo industry's troubled history.”

Both the investigative series and the editorial show a lack of understanding of flight safety and statistics. The scare language in both is not supported by the quoted statistics, most of the relevant characterizations about flight safety and who is responsible for it are incorrect, and there is a failure to address relevant tradeoffs.

Once again, the print media has shown that it can write well but is unable to refrain from writing about topics it does not understand.

Here are some excerpts from the editorial, along with my comments.

Air cargo service in America is like a ticking time bomb. One day a cargo plane -- with an exhausted pilot or mechanical problem -- will crash into a crowded mall or school and kill scores of people. The Federal Aviation Administration mustn't wait for such a tragedy to jump-start it into action.

This kind of statement is characteristic of the investigative report and the editorial. Slim possibilities are blown out of proportion and presented as emergencies.

It is virtually certain that, sooner or later, a mall or school crash will occur and kill scores (“Scores” means large numbers) of people. It also is virtually certain, assuming that the Miami Herald remains in business, that one of its delivery trucks will eventually crash into a group of grade school children, somewhere, killing scores of them. We can eliminate this possibility by closing down the Miami Herald.

What counts is the probability of such an air cargo crash and the cost of reducing it, not that such a crash is possible. The editorial’s own statistics suggest that the probability is small enough so that saving lives with scarce resources warrants using those resources elsewhere.

Given air cargo's track record -- one fatal crash per month on average since 2000 -- the FAA should tighten the industry's safety rules and oversight now, particularly for small cargo carriers that have the worst accident rates.

Air cargo is the deadliest segment of commercial aviation. Sixty-nine fatal crashes since 2000 have taken 85 lives.

The editorial fails to note the large number of air cargo flights. Nor does it strike the editor that sixty-nine fatal crashes since 2000 taking 85 lives implies a very low number of fatal crashes per year and a fatality rate of less than 2 per crash. This fatality rate per crash probably is far lower than for cars, bicycles, boating, etc.

The data implies an average fatality rate of less than 2 per fatal crash. This suggests the probability of crashing into a crowded mall or school and killing scores is very small. Why didn’t the Editor point this out? Two possibilities are that the Editor didn’t realize it and that the Editor did realize it but decided objectivity doesn’t sell enough papers. Neither possibility reflects well on the Editor or the Miami Herald.

Given air cargo's track record -- one fatal crash per month on average since 2000 -- the FAA should tighten the industry's safety rules and oversight now, particularly for small cargo carriers that have the worst accident rates.

Isn’t it true that the smallest air cargo carriers can be expected to have the lowest fatality rates per fatal crash? It might make sense (why might it not?) to spend more money on preventing airline crashes, where the fatality rate per fatal crash is often in the hundreds. What is the number of fatalities per year from the airlines? Is it higher than from air cargo? Where is the cost-benefit discussion?

The industry is hampered by aging planes, tight deadlines and overnight flights. Pilots fly under extreme pressure to meet tough deadlines. Many get paid only after the delivery.

Pilots often fly in hazardous weather, sometimes without enough sleep. Many fly planes that have mechanical problems, which is unthinkable in passenger flights.

Despite the problems, fewer FAA safety rules apply to cargo planes than passenger planes. For example, cargo pilots can fly 40 percent more hours per year; they are not required to have eight hours' rest 24 hours prior to a flight, as are passenger pilots. Small cargo planes aren't required to have ''black boxes'' that help determine the cause of a crash and can lead to safety improvements.

Old planes taken proper care of are safe. Overnight flights are safe and are common in airline schedules. Pilots are usually under pressure to meet deadlines, air cargo and airlines.

Pilots do not often fly in hazardous weather. They do often fly in weather. Very few pilots, air cargo or otherwise fly with mechanical problems that are serious. FAA regulations already prohibit such flying.

Perhaps FAA safety rules should be less restrictive for air cargo, because the consequences are less serious and the cost/benefit ratio favors less restrictive rules. Black boxes are desirable, but are they worth the cost?

Lax enforcement only worsens safety concerns for cargo carriers. In many cases, the FAA allows cargo firms to continue flying despite egregious maintenance lapses and crash histories. For example, the FAA has yet to ground one firm with four crashes involving seven deaths since 2002. That Ohio firm, Grand Aire Express, and its affiliate TriCoastal Air Inc. have had three dozen accidents or incidents since the 1980s, according to FAA records.

In 2000, the FAA fined Grand Aire $290,000 for operating a plane for 20 days without repairing a known problem and for other maintenance failures. Yet the companies continue to fly and accidents continue to happen. The latest crash took a pilot's life in February.

Remember the Chicago airline crash where an engine fell off on takeoff? That was due to two things. First, improper maintenance procedures. Second, pilot training that called for procedures that were inappropriate for the loss of an engine (as opposed to loss of an engine’s power). Did the FAA shut down the airline? Does the Herald’s Editor think the airline should have been shut down? That single, avoidable, crash killed more people than the sum total of air cargo fatalities, 85, quoted in the Herald’s editorial.

Grand Aire Express and TriCoastal do sound bad, and they may be. However, Luck plays a role, too. What is the probability that one properly run air cargo firm would suffer this kind of record from bad luck? What is the probability that at least one air cargo firm among many would suffer this kind of record from bad luck? I haven’t done the analysis, but neither has the Herald. I guarantee you the probability is much higher than you think (how does (1-p)^n behave?). Looking for the worst accident or fatality rate and implying that it is wholly due to poor maintenance, etc., is bad logic and bad statistics. If the Herald’s Editor and reporter do not know this, they are incompetent. If they do, they are dishonest.

The FAA says it has to be cautious before grounding planes because of the threat of lawsuits. Yet the cost of lawsuits, improved safety and more stringent enforcement should be weighed against the potential loss of life in the sky and on the ground.

Finally a tradeoff is considered. But, as usual, the cost to the industry and consumers of saving a few lives is ignored. One would have thought that this was the important tradeoff.

By pilot skill or luck, that crash hit a vacant lot and exploded just short of an industrial park west of Miami International Airport. The crash claimed five lives, including one on the ground. The toll could have been much worse. Had the plane come down just a little to the south, the plane could have plunged into the Mall of the Americas. Had the flight gone east, it could have exploded in dense residential neighborhoods or downtown Miami.

The fact that this crash and all the others quoted in the Editorial did not come down into a Mall and did not explode in a dense residential neighborhood or city says something about the probability of such an occurrence. Possibilities do not necessarily appreciable risks make.

Yet there can be no excuse for indifference to the threat of a potentially disastrous air-cargo crash. The FAA should take seriously its obligation to prevent death from raining down from our skies.

And, here, we have the final platitude. If the probability of a potentially disastrous air cargo crash is low enough and the cost of reducing it further is expensive enough, no effort should be made to reduce the probability further. “Death raining down from our skies” is a nice line, but irrelevant. Come to think of it, that is what the Herald’s investigative report and Editorial are; irrelevant. No, that is wrong. They are disasters themselves. To the extent the public and Government assigns resources as the Herald advocates, the Herald probably will cause an increase in the death rate due to the likelihood that assigning the same level of resources elsewhere would save more lives.

No comments: