A320 hull losses: Lies, damned lies and statistics

From:         Pete Mellor <pm@csr.city.ac.uk>
Date:         17 Jun 94 23:59:08 
View raw article
  or MIME structure

This thread of the discussion was originally started by Wesley Kaplow 
<kaploww@cs.rpi.edu> in RISKS DIGEST 16.15 under the title: 
"Does it matter why A3??'s have a poor record?" 

To recap, Wesley said (without citing a source): 

> Already, Airbus Industry has lost more planes per delivered plane
> than other major aircraft manufacturer in the past 3 decades (Lockheed,
> Boeing, MD). 

I contradicted this in RISKS 16.16, citing a table of statistics from an 
article entitled ``Der Traum von Total Sicherheit'' ["The Dream of Total 
Safety"], in the German magazine Focus, 38, 1993, pp18-21, and Wesley has 
since agreed that his statement requires support (see his follow-up mailings). 
The table was as follows:- 

>Aircraft         No. in     Hulls      % Losses 
>Type             Service    Lost              
>
>DC-9/MD-80       2065       68         3.29  
>Boeing 727       1831       62         3.39  
>Boeing 737       2515       57         2.27  
>Boeing 747       988        22         2.23  
>DC-10            446        21         4.71  
>Airbus A300/310  636        7          1.10  
>Airbus A320      411        4          0.97 

Focus magazine cited "Luftfahrtindustrie" (NOTE: not "Lundfahrtindustrie", 
as I originally transcribed it) as the source. 

Since then, I have been jumped on from a great height by several RISKS 
contributors who have accused me of abusing statistics. Since this is not 
something I do deliberately, I would like to make the following points 
(taking into account the various objections raised by those who have 
written to me):- 

1. I am well aware that the statistics above are incomplete. They do not 
   allow for the total operating time of each type. They do not distinguish 
   between losses due to on-board system failure and losses due to other 
   causes which could not possibly be blamed on the manufacturer (e.g., 
   the Lockerbie bombing, the Vincennes shoot-down). They do not take  
   account of wear-out and natural retirement, so that the number shown 
   may be the "number sold" and not the "number in service". I quoted them 
   because they were all I had at the time (while being acutely aware of 
   their imperfections). 

2. Wesley's original statement *is*, however, refuted by these statistics 
   *provided they are correct* (see point 3). A secondary question arises: 
   "Is this a meaningful measure of the safety of a type of aircraft?" 
   I will return to this in point 4. 

3. Peter Ladkin pointed out to me that the source name that I had originally 
   misread as "Lundfahrtindustrie" and assumed to be some official body 
   which records air accident statistics, is in fact "Luftfahrtindustrie" 
   (well, it was in small print! :-) and means simply "Air Travel Industry". 
   In other words, the source cited by Focus magazine is totally vague, and 
   (as Peter said) about as authoritative as "I read it on the net"! :-) 
   The statistics I naively quoted therefore need substantiation. 

4. What would convince Joe Public that a given aircraft type was safe to 
   fly? There are several possible measures of the safety of an aircraft 
   design (note: I do *not* pretend that this list is exhaustive):- 

   a) Deaths per passenger mile on the given type. This is used by the 
      aircraft manufacturing industry. Conclusion: air travel is the safest 
      way of going anywhere. 

   b) Deaths per passenger *hour* on the given type. This makes flying 
      about as safe as driving, but the risk would seem to be tolerable, 
      since a probability of 10^-4 per year of dying in a road accident 
      doesn't seem to worry most people (figures based on official UK 
      statistics). 

   c) Crashes (i.e., hull write-off) per revenue flight hour. This is used 
      by the certification agencies (FAA, JAA, etc.) when awarding an 
      airworthiness type certificate. The target is a maximum probability 
      of loss of aircraft of 10^-6 per flight hour due to *all* causes. 
      Historically, statistics show that *system-related* causes account for 
      1 in 10. The conservative assumption that there are 100 critical 
      systems on board then leads to the famous 10^-9 requirement for 
      probability of failure of an individual flight-critical system. 

   d) Crashes per cycle (take-off plus landing). 

   e) Crashes per example delivered (which is where we came in! :-) 

   f) Passenger deaths per cycle. 

   g) Serious incidents per flight hour or per cycle. (Q: "How many accidents 
      has the A320 had?"  A: "Five - You forgot about Lille, where an A320 
      landed on top of a Mooney, taking off both its wings and the empennage, 
      and collapsing the A320's front gear. Since nobody was hurt, it doesn't 
      count, or does it?") 

   The whole picture is confused by the fact that the public perception 
   of risk is biased *against* rare events that kill lots of people, and 
   less against common events that kill a few. (In assessing any event that 
   loses the aircraft, you must assume the worst case: that you kill everyone 
   on board. If you crash a car, it's just you and the guy you hit!) 

   I don't pretend to give an answer here, simply raise a few pertinent 
   questions, whose answers (IMHO) are far from obvious. 

5. A fairer comparison would be between the A320 and competing aircraft 
   *of the same generation*. I would like to thank Robert Dorsett for 
   the following:- 

   757  = 0 in eleven years.
   767  = 1 in twelve years. (Lauda)
   A320 = 4 in five years. (Air France, Indian Airlines, Air Inter, Lufthansa)

6. Given that all the statistics above are deficient (basically, they lack 
   an exposure time base), they do still tell us *something*. (In considering 
   a fleet above a certain size, we could assume roughly the same operating 
   hours per day for each example, and things like maintenance time, etc., 
   would average out.) We could *tentatively* conclude that the A320 is a 
   long way from being a flying coffin, but also a long way from being the 
   safest aircraft ever, or even as safe as it should be, given its modernity. 
   The public perception of the A320 seems to be that it is the most 
   dangerous thing that ever left the ground. IMHO this is wrong, and we 
   should be careful not to spread false alarm. 


There are, of course, better statistics (e.g., from Flight International) 
and I shall attempt to locate a few. The best come from the air insurance 
industry, but I am not sure that I can get my grubby paws on those for 
reasons of confidentiality. 

In the meantime, if anyone knows of a good source ... :-) 

Also, how can I phrase an argument to convince my mother that I stand a 
greater chance of being run over crossing St. Johns Road while walking from 
Farringdon tube station to the University than I do of dying in an air crash? 
Then I won't have to make long distance 'phone calls from the airport in 
B*m***k Egypt to tell her we landed safely every time I go to a conference! :-) 

Peter Mellor, Centre for Software Reliability, 
City University, Northampton Square, London EC1V 0HB 
Tel: +44 (71) 477-8422, Fax.: +44 (71) 477-8585, 
E-mail (JANET): p.mellor@csr.city.ac.uk 

[Original posted to RISKS] 
-----------------------------------------------------------------------------