Task 2.1.1: Updated review of AMJ 25.1309 (ACK)

From:         Pete Mellor <pm@csr.city.ac.uk>
Date:         25 Jul 93 00:22:05 PDT
View raw article
  or MIME structure

Dear Jen, 

Please find following the latest version of my review of AMJ 25.1309, for 
inclusion in the state-of-the-art report. I have taken account of your 
pencilled comments on the first draft (in most cases! :-) and included 
some important points which arose in discussion with Lorenzo. 

I have taken the liberty of broadcasting this to the airliners list.
Any comments from its readers are more than welcome. 

Peter Mellor, Centre for Software Reliability, City University, Northampton 
Sq., London EC1V 0HB, Tel: +44(0)71-477-8422, JANET: p.mellor@csr.city.ac.uk 
-----------------------------------------------------------------------------

                 Review of AMJ 25.1309 and Related Documents 

Summary 

The "Joint Airworthiness Requirements" (JAR) contain the European requirements 
for any design of aircraft to be awarded "type certification". Part 25 applies 
to "large aeroplanes". These requirements were originally established jointly 
by the Airworthiness Authorities of France, Germany, The Netherlands and the 
UK. JAR-25 merely states requirements for certification that a given aircraft 
design is safe, usually with no guidance on how a manufacturer is to 
demonstrate that the design complies with them. JAR-25 must therefore be 
applied together with a set of "advisory material", which interprets the 
requirements, and gives guidelines for demonstrating compliance. 

Paragraph JAR 25.1309 covers airborne systems and the effects of their failure 
on the whole aircraft and its occupants. It is supplemented by AMJ 25.1309 
"System Design and Analysis", which is one document in the "Advisory Material 
Joint" series. It came into force at JAR-25 change level 13. At previous change 
levels, a similar role was played by ACJ No. 1 to 25.1309 ("Advisory Circular 
Joint"). 

The equivalent regulations in the USA are the Federal Aviation Regulations 
(FAR), and JAR-25 is very similar to FAR-25. There is also a correspondence 
between the advisory material issued by the European and US authorities. In 
particular, AC 25.1309-1A ("Advisory Circular") corresponds to AMJ 25.1309. 

This review covers the advisory material which supplements JAR 25.1309, and 
should be considered together with the review of JAR-25. 

AMJ 25.1309 clarifies and quantifies the notions of risk which JAR 25.1309 
introduces. It classifies failure conditions by their severity, and specifies 
a maximum permissible probability (both in terms of categories such as 
"improbable" and "extremely improbable", and in terms of numerical 
probabilities such as "10^-9 per flight hour") with which failure conditions 
in each class may arise. With regard to failure conditions which could arise 
from the anomalous behaviour of the software in a programmable system, it 
states that their probabilities cannot be assessed, and invokes a set of 
process guidelines (RTCA DO-178A) for the development of the software. 

ACJ No. 1 to 25.1309 defines the same failure condition severities and 
probability levels as AMJ 25.1309, but does not contain as much information.
In particular it makes no reference to systems containing software. There are 
several other ACJs to JAR 25.1309, all very brief. 


1. Introduction 

Originating organisation: JAA (Joint Airworthiness Authority) 

Numbers: AMJ 25.1309 (Advisory Material Joint relating to JAR 25.1309) 
This review also covers ACJ Nos. 1 to 8 to JAR 25.1309 (Advisory Circulars Joint). 

Status: Guidelines for means of demonstrating compliance with JAR 25.1309 
for aircraft type certification. 

Version: Amendment 90/1  

Date: Effective 11.05.90

Title: System Design and Analysis 

Language: English 

Purpose of AMJ 25.1309: 

(Quote from Section 1: Purpose) 

This AMJ describes various acceptable means for showing compliance with the 
requirements of JAR 25.1309 (b), (c) and (d). These means are intended to 
provide guidance for the experienced engineering and operational judgement 
that must form the basis for compliance findings. They are not mandatory. 
Other means may be used if they show compliance with this section of the 
requirements. 

(End quote) 

Compliance with JAR-25, and particularly with JAR 25.1309 must be demonstrated 
before any aircraft design is awarded an airworthiness type certificate. 

AMJ 25.1309 contains the following sections:- 

1. Purpose   (See above) 
2. Reserved  (for future use, should anyone think of a reason for having a 
              section 2! This may be included to preserve a similar layout 
              to AC 25.1309-1A.) 
3. Applicability 
4. Background 
5. The Fail-Safe Design Concept 
6. Definitions 
7. Discussion 
8. Acceptable Techniques 
9. Qualitative Assessment 
10. Quantitative Assessment 
11. Operational and Maintenance Considerations 
12. Step-by-step Guide 
Fig. 1. Relationship between probability and severity of failure condition 
Fig. 2. Depth of Analysis Flowchart 

AMJ 25.1309 clarifies and quantifies the notions of risk which JAR 25.1309 
introduces. It classifies failure conditions by their severity, and specifies 
a maximum permissible probability (both in terms of categories such as 
"improbable" and "extremely improbable", and in terms of numerical 
probabilities such as "10^-9 per flight hour") with which failure conditions 
in each class may arise. With regard to failure conditions which could arise 
from the anomalous behaviour of the software in a programmable system, it 
states that their probabilities cannot be assessed, and invokes a set of 
process guidelines (RTCA DO-178A) for the development of the software. 


ACJ No. 1 to JAR 25.1309 " ... consists of interpretive material and acceptable 
means of compliance, together with definition of the terms associated with 
probabilities." It defines the same failure condition severity categories and 
corresponding maximum permissible probabilities of occurrence as AMJ 25.1309, 
but without some of the more detailed explanation and description of methods. 
In particular, it does not mention software, and does not invoke RTCA DO-178A. 

ACJ Nos. 2 to 8 to JAR 25.1309 are all extremely short, and simply raise 
particular details that should be taken into account when assessing compliance. 

The ACJs are listed and summarised in an appendix to this review. The versions 
of the ACJs reviewed apply to JAR-25 at change 11. The latest JAR is change 13, 
at which level AMJ 25.1309 superseded ACJ No. 1 to JAR 25.1309. Later versions 
of ACJ Nos. 2 to 8 (if any) were not available for review. 

The rest of this review deals solely with AMJ 25.1309, associated with JAR-25 
change 13. 


2. PLANT SAFETY CONTEXT

2.1. Relevant Safety Legislation/Regulation

- What laws apply to the industry (e.g. EC Health and Safety Directives) 

National statutes govern civil aviation in each country, and each country 
has its own regulatory authority to enforce legal requirements and other 
national regulations on manufacturers, carriers, and airport and air traffic 
control autorities. 

- Is there explicit regulation and licensing? 

Yes. Without compliance with JAR-25, a manufacturer cannot obtain "type 
certification", which is essential before any carrier may operate the given 
type of aircraft in a civil transport fleet. 

- if so what body regulates safety in the industry? 

The various national civil airworthiness authorities establish regulations 
and monitor all aspects of safety in civil air transport in their own 
countries. For the purposes of drafting regulations and awarding type 
certificates, the airworthiness authorities of France (Direction Generale de 
l'Aviation Civile - DGAC), Germany (Luftfahrt Bundesamt - LBA), The Netherlands 
(Rijksluchvaartdienst Directie Luchvaartinspectie - RLD) and the UK (Civil 
Aviation Authority - CAA) came together in the late 1980s to form the Joint 
Airworthiness Authority (JAA), and established the JAR as the common set of 
regulations for all four countries (with minor national variants). 

In the USA, the Federal Aviation Administration (FAA) performs the same 
function according to the Federal Aviation Regulations (FAR). 


- How does the company interact with the regulators,
  and at what stages during the development of a safety-related system?

Methods of demonstrating compliance are agreed between the manufacturer and 
the authorities, making use of the associated advisory material. The two 
parties then work together closely throughout the whole of development from 
the concept phase onward to agree on, and apply, acceptable means of 
demonstrating compliance with the requirements of JAR-25, and particularly 
those of JAR 25.1309. 

AMJ 25.1309 states explicitly in several places that this close cooperation 
should be established as early as possible, since none of the acceptable means 
which it describes is mandatory, and hence the means for every system must be 
agreed in advance with the authority. This applies from the functional 
hazard analysis (which establishes the severity of every failure condition), 
through the more detailed analyses of individual hazardous and catastrophic 
failure conditions, down to the actual running of tests or provision of other 
evidence to establish the safety case by showing that each failure condition 
has an acceptably low probability of occurrence. 

For example, AMJ 25.1309 section 8 "Acceptable Techniques" states: 

(Quote) 

The methods outlined in this section provide acceptable techniques, but not 
the only techniques, for determining compliance with the requirements of 
JAR 25.1309 (b), (c) and (d). Other comparable techniques exist and may be 
proposed by an applicant for use in any certification programme. Early 
agreement between the applicant and the Certifying Authority should be reached 
on the methods of assessment to be used. 

(End quote) 


2.2. Applicable Standards

JAR-25 is a set of mandatory requirements for safety in aircraft design. 
In addition to Part 25, JAR-AWO applies to types of aircraft intended for 
all-weather operation, and JAR-E applies to engine certification. 

Minor national variants of these standards are retained by the various JAA 
members. The US requirements for design safety (FAR-25) are very similar. 

Paragraph JAR 25.1309 states the requirements concerning the effects of 
failure of avionics systems on the whole aircraft. Compliance with these 
is at the heart of the manufacturer's safety case. 

The AMJ and ACJ documents clarify the requirements and describe means of 
demonstrating compliance. The means described are not mandatory, but may be 
used in any particular certification by agreement between the manufacturer and 
the airworthiness authorities. 

The software in a programmable system must be developed according to the 
process guidelines described in RTCA DO-178A, (Requirements and Technical 
Concepts for Aviation, document 178 "Software Considerations in Airborne 
Systems and Equipment Certification"). Almost identical guidelines are referred 
to in Europe as EUROCAE ED-12A (European Organisation for Civil Aviation 
Equipment). AMJ 25.1309 and AC 25.1309-1A) explicitly invoke RTCA DO-178A, 
or later revisions thereto. (The version currently in force is version B.) 

In addition, each aircraft manufacturer will have its own internal company 
standards. For example, Airbus Industrie applies "ABDs" (Airbus Directives) 
both to in-house development and to procurement from suppliers. ABDs clarify 
and expand the certifying authorities' regulations to an even greater level 
of detail than the advisory material, or other standards invoked, such as 
RTCA DO-178A. Where certification is sought from several authories (e.g., JAA 
and FAA), ABDs will normally be aimed to guarantee compliance with the most 
stringent regulatory requirement imposed by any of the authorities. 

Manufacturers will normally be certified according to ISO-9000. 



2.3 Approach to Safety Assurance

The general principle applied (AMJ 25.1309 section 4a) is that: "... an inverse 
relationship should exist between the probability of loss of function(s) 
or malfunction(s) ... and the degree of hazard to the aeroplane and its 
occupants arising therefrom." 

The AMJ defines several levels of severity, together with corresponding 
levels of probability of failure (loss of function, etc.), expressed both 
quantitatively (numerical probabilities) and qualitatively (probability 
categories). 

In addition, the "fail-safe design concept" is recommended. 

A functional hazard analysis is recommended as a preliminary step to identify 
the possible failure conditions and their severity, and those graded as 
"hazardous" or "catastrophic" should be subjected to more detailed analysis to 
show that their occurrence is acceptably improbable. In particular, numerical 
probabilities should be used when assessing the probability of a catastrophic 
failure condition, but should not be the sole means of assessment. 


2.3.1 Defining Safety

a) Describe how the safety limits are set.

   - What are the tolerable risks for: plant personnel and the public? 

The following is adapted from AMJ 25.1309 4a:- 

The acceptability of a design in terms of the probability of it causing death 
is based on historical statistics of serious aircraft accidents (resulting in 
fatalities) which were due to operational and air-frame related causes. A 
conservative figure for the frequency of these is 1 per million hours of 
flight. (Recent statistics show a great improvement on this figure.) 
About 10% of these accidents can be attributed to failure conditions arising 
from the aeroplane's systems. On the principle that new designs should not be 
allowed a higher probability than already exists of causing serious accidents, 
the maximum frequency of serious accident due to all failure conditions is set 
at 1 per ten million hours. It is assumed that there are 100 possible failure 
conditions on an aircraft which can prevent continued safe flight and landing. 
(This assumption is somewhat arbitrary. It is similar to assuming that there 
are 100 safety-critical systems on each aircraft. To take a real example, there 
are around 70 safety-critical systems on the A320.) The allowable probability 
of failure is then apportioned equally among the 100 failure conditions, giving 
a maximum permissible frequency of occurrence of each failure condition of one 
per billion hours of flight, or the famous probability of catastrophic failure 
of 10^-9 per hour of flight. 

This establishes an upper limit for "Extremely Improbable" as used in 
JAR 25.1309 to characterise any failure condition "... which would prevent 
continued safe flight and landing". 

Higher probabilities of failure are permitted for less severe failure 
conditions. 


   - Are there limits for different grades of injury/accident ? 

Four grades of Failure Condition are defined as follows in AMJ 25.1309, 
section 6j:- 

(1) MINOR: Failure Conditions which would not significantly reduce aeroplane 
safety, and which involve crew actions that are well within their capabilities. 
Minor failure conditions may include, for example, a slight reduction in 
safety margins or functional capabilities, a slight increase in crew workload, 
such as routine flight plan changes, or some inconvenience to occupants. 

(2) MAJOR: Failure Conditions which would reduce the capability of the 
aeroplane or the ability of the crew to cope with adverse operating conditions 
to the extent that there would be, for example, a significant reduction in 
safety margins or functional capabilities, a significant increase in crew 
workload or in conditions impairing crew efficiency, or discomfort to 
occupants, possibly including injuries. 

(3) HAZARDOUS: Failure Conditions which would reduce the capability of the 
aeroplane or the ability of the crew to cope with adverse operating conditions 
to the extent that there would be: 
(i) A large reduction in safety margins or functional capabilities; 
(ii) Physical distress or higher workload such that the flight crew cannot be 
relied upon to perform their tasks accurately or completely; or 
(iii) Serious or fatal injury to a relatively small number of the occupants. 

(4) CATASTROPHIC: Failure Conditions which would prevent the Continued Safe 
Flight and Landing. 

Minor failure conditions may be "Probable". 

Major failure conditions may be at most "Remote". 

Hazardous failure conditions may be at most "Extremely Remote". 

Catastrophic failure conditions may be at most "Extremely Improbable". 

(The probability categories are defined in AMJ 25.1309, sections 9e and 10b.) 



   - Are the frequencies of such events expressed qualitatively or 
     quantitatively? (e.g. 0.1/year, or `unlikely', `remote', etc.) 

Both qualitative and quantitative assessments are recommended. 

The probability categories are defined qualitatively as follows 
(AMJ 25.1309, 9e):- 

Probable: anticipated to occur one or more times during the entire operational 
life of a single aeroplane. 

Remote: Unlikely to occur to each aeroplane during its total life but which 
may occur several times when considering the total operational life of a 
number of aeroplanes of the type. 

Extremely Remote: Unlikely to occur when considering the total operational 
life of all aeroplanes of the type, but nevertheless has to be considered 
as being possible. 

Extremely Improbable: So unlikely that they are not anticipated to occur 
during the entire operational life of all aeroplanes of one type. 


The probability categories are defined quantitatively as follows 
(AMJ 25.1309, 10b):- 

Probable: probability greater than of the order of 1 x 10^-5. 

Remote: probability of the order of 1 x 10^-5 or less, but greater than 
of the order of 1 x 10^-7. 

Extremely Remote: probability of the order of 1 x 10^-7 or less, but 
greater than of the order of 1 x 10^-9. 

Extremely Improbable: probability of the order of 1 x 10^-9 or less. 


Notes:- 

1. Probabilities are usually quantified "... in terms of acceptable 
numerical probability ranges for each flight hour, based on a flight of mean 
duration for the aeroplane type. However, for a function which is used only 
during a specific flight operation; e.g., take-off, landing, etc., the 
acceptable probability should be based on, and expressed in terms of, the 
flight operation's actual duration." (Quoted from AMJ 25.1309, 10b) 

2. The term "Improbable" is used to include both "Remote" and "Extremely 
Remote" in both numeric and non-numeric assessment. 

3. The phrase "of the order of" is used to indicate that a numerical  
assessment will not usually be precise. 



b) How is this transcribed into design targets for the plant? 

   - Is there a direct link between plant failures and injury?

Yes. The severity categories of failure condition are defined partly in terms 
of the degrees of injury. Each severity category has a maximum permissible 
probability of occurrence for the failure conditions within it. 

   - If indirect, how are the plant safety targets established? 

N/A. 



2.3.2 Implementing Safety

a) How are the plant hazards identified ? 

   - Is some systematic procedure followed? 

Yes. A functional hazard assessment is recommended to identify every failure 
mode of every system and classify the resulting potential faliure condition 
as minor, major, etc. The procedure for assessing the probability of failure 
conditions is shown in flowchart form in AMJ 25.1309 Fig. 2. 


b) How are the risks of these hazards assessed? 

On the basis of operational and engineering judgement, and the results of 
testing and previous operational experience with similar equipment. 

According to AMJ 25.1309 section 8 and the flowchart in Fig. 2, any failure 
condition shown to be minor need not be analysed further. 

Major failure conditions should be assessed qualitatively to be "remote". 
Provided the system has low "complexity" and is similar in its relevant 
"attributes" to those on other aircraft, then design and installation 
appraisals, and satisfactory service history will suffice. FMEA or fault 
tree analysis should be used for complex systems with functional redundancy, 
to rule out the existence of common cause failures, and to show that there 
are no effects on other functions. 

Hazardous and catastrophic conditions should be assessed to be "extremely 
remote" and "extremely improbable", respectively. This is usually done by a 
combination of qualitative and quantitative assessment, but catastrophic 
failure conditions should not be assessed by quantitative methods alone. 
For simple conventional systems, assessment can be made using "experienced 
engineering judgement", based on the degree of redundancy and on satisfactory 
service history of similar equipment, where this exists. 

The terms "complex" and  "conventional" are defined in AMJ 25.1309 section 6 
as follows:- 

COMPLEX: Applicable to systems whose architecture and logic are difficult to 
comprehend without the aid of analytical tools, e.g., Failure Modes and 
Effects Analysis, Fault Trees, Reliability Block Diagrams. 

CONVENTIONAL: An attribute of a system is considered to be conventional if 
it is the same as, or closely similar to, that of previously-approved systems 
that are commonly used. 
 

A failure condition resulting from a single failure mode of a device cannot 
be accepted to be extremely improbable, except in very unusual cases. 

AMJ 25.1309 section 7i states that: "In general, the means of compliance 
described in this AMJ are not directly applicable to software assessments 
because it is not feasible to assess the number or kinds of software errors, 
if any, that may remain after the completion of system design, development 
and test. RTCA DO-178A and EUROCAE ED-12A, or later revisions thereto, provide 
acceptable means for assessing and controlling the software used to program 
digital-computer-based systems." In practice, this means that a reliability 
of 1 is apportioned to the software when assessing the risk from the failure 
of any programmable system. 

What the AMJ recommends, in effect, is:- 

For non-critical systems, use judgement. 

For simple systems, use judgement. 

For conventional systems, appeal to previous satisfactory service record 
of something similar. 

For the rest, use numerical probabilities (but not as the sole means of 
assessment). 

By any criteria, computer-based systems are complex and unconventional, and 
can be critical (e.g., the A320 EFCS). Even if the hardware architecture is 
conventional (e.g., there is a close similarity between the dual-channel 
fail-passive computers in the A320 EFCS and those in earlier non-critical 
systems in the A310 and other aircraft), the presence of different software 
immediately destroys any similarity between two computer-based systems. 


AMJ 25.1309 is therefore sending out contradictory messages:- 

On the one hand, it recommends that computer-based systems be assessed 
using numerical failure probabilities (since they are complex and 
unconventional). 

On the other hand, it states that the probability of anomalous behaviour of 
the software which is at the heart of such systems cannot be assessed at all, 
numerically or otherwise! 


This is a serious deficiency in the whole of the regulations as applied to 
airborne computer-based systems. 


c) What design approach is followed to reduce unacceptable risks?

   - plant redesign 

The detection of any unacceptably probable failure condition during functional 
hazard analysis would result in a modified design being proposed prior to 
certification. Detection during operation, e.g., by an accident investigation, 
would result in a post-certification modification and recertification. In 
some cases, modifications may be enforced by Airworthiness Directives. 


   - plant control systems 

Assessment of compliance with JAR 25.1309 (c) and (d)4 (warning information) 
is described in AMJ 25.1309 section 8g. Warnings must be "timely, rousing, 
obvious, clear and unambiguous" and must be issued in time for the crew to 
take corrective action. The failure monitoring and warning system should be 
reliable in not failing to give genuine warnings and in not giving excessive 
false alarms. 


   - independent plant safety systems 
     (mechanical protection, computer protection, etc.) 

The "fail-safe design concept" is recommended. (See below.) 


   - human operators and safety procedures. 

Procedures to be followed after a warning should be described in the Aeroplane 
Flight Manual. Certification Check Requirements (CCR) (see below) should not 
be used in lieu of a reliable failure monitoring and warning system. 

Quantitative assessment of the probability of crew error is not considered 
feasible. (AMJ 25.1309 8g(5)) 


   - what assumptions are made about the independence of the various
     safety systems (e.g.: fully independent) -if so how is this
     demonstrated?

The "fail-safe design concept" (AMJ 25.1309 section 5) requires that the 
failure of any single element or component must be assumed when assessing a 
system. FMEA or fault-tree analysis may be used to demonstrate independence. 



d) What policy is followed in reducing the plant safety risks? 


   - Does the plant have a safe shutdown state?

No. From decision point during take-off onward, certain systems must continue 
to function to allow continued safe flight and landing. 


   - How do you balance safety against availability 
     (e.g. shut-down, or degraded operation)?

Certain systems must be available in order to ensure safety. There is no 
trade-off. The fail-safe design concept recommends error-tolerance and a 
"designed failure path" to proceed in an orderly way to degraded operation 
following failure of a system. The manufacturer may draw up a Master Minimum 
Equipment List (MMEL) without which the aeroplane will not be allowed to 
take-off for safety reasons. 


   - What level of reliance is placed on the operators 
     (is safety-action automated or operator-dependent)? 

Certain systems will respond automatically to failure conditions or external 
events (e.g., wind gusts) which may affect safety, but in many cases safety 
depends on the crew responding correctly to warnings. Certain types and phases 
of flight are necessarily automated, e.g., category 3 landings (zero visibility 
and no decision height) using instrument landing systems (ILS). 


   - How much does the operator rely on automatically processed data?
     (filtering, expert systems, "glass" consoles) ? 

The crew rely heavily on information displayed in the cockpit by failure 
monitoring and warning systems. 



2.3.3 Maintaining Safety


   - Does the plant design explicitly consider the impact of maintenance
     and repair on safety? 

AMJ 25.1309 section 11 considers operation and maintenance. 


   - Are there explicit maintenance procedures? 

AMJ 25.1309 section 11 allows the manufacturer to define Certification Check 
Requirements (CCRs), which are periodic checks carried out by ground or flight 
crew to detect latent faults. Where these are done by flight crew, the 
procedures should be described in the AFM. Where they are done by ground crew, 
they should be made available to the carrier in time to be incorporated in the 
maintenance programme. 


   - What procedures are followed for a plant modification?

The manufacturer will apply to the airworthiness authority for approval of 
any modification. 


   - How is the impact of modification on plant safety assessed?

The recertification after a modification follows a subset of the procedures 
for original type certification. 



2.3.4 Assessing/Assuring Safety

a) How is the risk of the combined plant, control, protection systems and
   operational personnel assessed? 

By showing that all identified failure conditions have the appropriate 
probability of occurrence. This should then mean that the probability of 
loss of hull through all system-related causes is no greater than 10^-6 
per hour of flight. (See above.) 


b) For regulated industries, how is the plant safety justified to the
   regulators? 

By making out a safety case using criteria agreed upon in close consultation 
with the airworthiness authority during the development of the aircraft type. 


c) For non-regulated industries, is there an internal safety body that
   approves the safety of the plant? 

N/A 


d) Is there an established procedure for feed-back from the safety
   assessment to the designers? 

The designers are closely involved in the safety assessment. 



2.3.5 Methods and Techniques

The following specific assessment techniques are recommended in AMJ 25.1309:

Functional hazard assessment: A preliminary assessment of the design to 
identify possible failure conditions and grade their severity, taking into 
account external events (e.g., weather conditions) which may affect safety in 
combination with system failure. 

Installation appraisal: A qualitative appraisal of the integrity and safety 
of an installation, using expeienced judgement. 

Fault Tree Analysis: This may be applied qualitatively (AMJ 25.1309 section 9d) 
or quantitatively (AMJ 25.1309 section 10a). 

Dependence diagram: This is more commonly known as a reliability block diagram, 
and is similar to FTA, but is success-oriented rather than failure-oriented. 
It may be used qualitatively (AMJ 25.1309 section 9d) or quantitatively 
(AMJ 25.1309 section 10a).

Failure modes and Effects Analysis (FMEA): This may be applied qualitatively 
(AMJ 25.1309 section 9c) or quantitatively (AMJ 25.1309 section 10a).

Common Cause Failure Analysis: This is referred to in several places where 
the risk from failure of a given system is to be assessed. It is stated that 
Fault Tree Analysis or FMEA could uncover common cause failures. 



3. DEALING WITH DESIGN ERROR

The following sections describe what AMJ 25.1309 recommends under each main 
heading.

3.1. Fault Avoidance

Project Controls: Evidence must be presented to establish the safety of each 
system on the aircraft. 

Quality Control Systems: Experts drawn from the airworthiness authorities 
closely supervise design and development at all stages. The FAA uses the DER 
system for supervision, and a similar system of "authorised signatories" 
exists in the UK. 
  
Design Reviews, Inspections, etc.: These are recommended as part of 
establishing the safety case. See above. 

Staff Competency: AMJ 25.1309 makes no specific statement about this. However, 
aircraft designers would normally be expected to be chartered engineers in 
some field. Aircrew must be formally qualified on any type of aircraft before 
being allowed to fly it. This involves training in simulators and flight, 
followed by check flights with a qualified pilot to check the trainee's 
performance. No formal qualifications exist for maintenance crews, but they 
would be expected to pass through the training programme of either the carrier 
company or the aircraft manufacturer, and to be able to follow CCR and other 
procedures related to safety. 

Design Tools and Methods: Not specifically mentioned by AMJ 25.1309, other than 
the analysis procedures menti9oned above. 

Plant simulation: Simulation tests are mentioned as a means of establishing 
compliance with JAR 25.1309. 


3.2. Fault Detection

- Prototyping: Not explicitly mentioned. 

- Reviews, Inspections, HAZOP, etc.: Most of the assessment techniques 
recommended fall into this category. 

- Plant/system simulation: See above. 

- Testing: Mentioned as one of the means of demonstrating compliance with 
JAR 25.1309


3.3. Failure Toleration/Mitigation

AMJ 25.1309 section 5 describes the "fail-safe design concept", and other 
sections of the document refer to the use of redundancy. 

 - Design conservatism (safety factors): Section 5 (11) recommends the 
incorporation of safety margins. 

 - Claim limits on reliability: Single point failures may not be assessed 
as "extremely improbable". 

 - Fail-safe design: This is recommended, and involves:- 

1. Designed integrity and quality 
2. Redundancy or back-up systems 
3. Isolation to prevent failure propagation 
4. Proven reliability 
5. Failure warning 
6. Defined flightcrew procedures 
7. Checkability: the ability to check a component's condition. 
8. Failure containment to limit the impact of a failure 
9. Designed failure path to control and direct the effects of a failure. 
10. Error-tolerance 
11. Margins or factors of safety

 - Design diversity: Not specifically recommended. 

 - Treatment of common cause failure (CCF): Recommended in several places, 
but not as an activity on its own. 

 - use of Beta factors in design: Not mentioned. 


3.4 Incident Reporting and Analysis

This is not specifically covered by AMJ 25.1309, however:-  

 - Failure databases 

The airworthiness authorities maintain databases of equipment failures, and 
pilots may report incidents anonymously to a reporting forum. 

 - Incident analysis

Any aircraft accident is officially investigated to establish its causes, using 
data from the CVR and DFDR. 

 - Design reassessment 

Would occur frequently during the process of obtaining a type certificate. 

 - Design modification 

Would be carried out in the light of functional hazard assessment and 
assessment of individual failure conditions. 


3.5 Assessment 

See section 2.3.5 above. 




4. 10^-9 and all that ... 

... the non-assessment of safety-critical avionics software. 

AMJ 25.1309 states that catastrophic failure conditions should be assessed as 
"extremely improbable", and that this assessment may be numerical. It is 
also concerned that systems which are "complex" (require FMEA or FTA in order 
to be understood) should be carefully assessed, and also those which are not 
"conventional". By any criteria, digital computer based systems are complex 
and unconventional, and so would be expected to be analysed carefully and 
numerically. However, AMJ 25.1309 specifically does not recommend 
probabilistic assessment of software, and instead invokes RTCA DO-178A, which 
is a set of process guidelines. 

In practice, this means that the software in a system is apportioned a 
reliability of 1 when the system is assessed. For example, the A320 EFCS 
is a safety-critical system. Its individual computers have a hardware 
failure probability of the order of 10^-3 to 10^-4. The individual computers 
are connected in such a way that the whole EFCS can be shown to have the 
required probability of failure of 10^-9, but this is with respect to hardware 
failure only. 

There would seem to be a good case for having AMJ 25.1309 impose a claim 
limit on the reliability of software in such systems. 


APPENDIX: Advisory Circulars to JAR 25.1309 

The following ACJs apply to JAR-25 at change 11. 

ACJ No. 1 to JAR 25.1309 " ... consists of interpretive material and acceptable 
means of compliance, together with definition of the terms associated with 
probabilities." It contains the same failure condition severity categories and 
associated probabilities of occurrence as AMJ 25.1309, but without some of the 
detailed explanation and discussion. It does not mention computer-based systems 
specifically, and does not invoke RTCA DO-178A. 

ACJ No. 2 to JAR 25.1309 states simply: "The effects of fluid or vapour 
contamination, due either to the normal environment or accidental leaks or 
spillage, should be taken into account." (Applies to JAR 25.1309(a))

ACJ No. 3 to JAR 25.1309 states simply: "The effects of mechanical damage or 
deterioration including short circuits or earths caused by such damage, in 
particular the failure of an earth connection should be taken into account." 
(Applies to JAR 25.1309(b)) 

ACJ No. 4 to JAR 25.1309 states that each source of electrical supply 
(generator or battery) should be provided with a warning light to give 
the crew immediate warning of failure of its output. (Applies to JAR 25.1309(c))

ACJ No. 5 to JAR 25.1309 states that, for compliance with JAR 25.1309(e), 
all possible combinations of electrical power source failure should be 
considered, except those that are shown to be extremely improbable. This 
should include loss of all main generated power, and emergency supplies 
should be provided for essential services. These emergency supplies should 
be mechanically and electrically isolated from the normal system, so that no 
single failure can affect both. 

ACJ No. 6 to JAR 25.1309 states that the separation of redundant electrical 
and hydraulic circuits should be such as to minimise the probability of a 
single failure affecting all. (Applies to JAR 25.1309(e)) 

ACJ No. 7 to JAR 25.1309 states that for aeroplanes for which 
"two-power-units-inoperative performance is scheduled" (i.e., they are 
expected to keep flying after two engines out of three or four have failed) 
sufficient services should remain to enable continued safe flight and landing, 
although a certain degradation in some services may be expected. 
(Applies to JAR 25.1309(e)(3))

ACJ No. 8 to JAR 25.1309 states simply: 
"1 The reliability of each warning system should be compatible with the 
reliability of the system for which it provides a warning. 
2 Each warning system should be designed so as to minimise unnecessary 
warnings." (Applies to JAR 25.1309(c)) 

It will be seen that most of the ACJs to JAR 25.1309 are fairly trivial, and 
address specific small details, apart from ACJ No. 1, which was superseded by 
AMJ 25.1309 at JAR change 13. 

----------------------------------------------------------------------------