Re: N-version software compared to others

From:         Pete Mellor <pm@csr.city.ac.uk>
Date:         24 Jun 96 12:24:54 
References:   1 2 3
Next article
View raw article
  or MIME structure

Tom Speer <speer%do.edw@mhs.elan.af.mil> wrote on Mon Jun 24 08:49:28 1996:

> I looked into the pro's and con's of N-version programming, and ran
> accross some interesting studies.  The one that impressed me the most
> was one in which a couple of universities teamed up to provide something
> like 16 independent programmer teams to code software from a common
> spec.

The reference is:-
Knight J.C., Leveson N.G.: "An experimental evaluation of the assumption
of independence in multiversion programming", IEEE Transactions on Software
Engineering, Vol. SE-12, (1), Jan. 1986, pp 96-109.

Tom's summary of the experiment is pretty accurate, except that there
were 27 different programs written from the same spec. These were written
independently by 27 different students (or teams?) from two different
universities, and 1 million test cases were applied, and the results
compared with the "golden" version.

Knight and Leveson's basic finding (that at a 98% confidence level, the
independently written versions did not *fail* independently) effectively
put a big nail in the coffin of N-version as a means of obtaining levels
of reliability of the order of 1-10^9 per flight hour.

Pete Mellor
-------------------------------------------------------------------------