What changed in Part-ORA?

MANAGEMENT SYSTEM – PERSONNEL IN SUPPORT OF AN INITIAL, RECURRENT OR SPECIAL

EVALUATION

The following persons from the organisation operating the FSTD should be present to support the

evaluation:

(a)

a pilot with a valid class or type rating for the aircraft simulated, and with sufficient knowledge

and flying experience on such aircraft, unless otherwise agreed with the competent authority;

and

(b)

FSTD support staff to assist in the running of tests and the operation of the instructor’s station.

PROCEDURE FOR MANAGEMENT OF AN EQUIPMENT SPECIFICATION LIST (ESL)

The procedure for management of an equipment specification list (ESL) should be part of the

management system of the organisation operating the FSTD and include at least the following:

(a)

the position(s) of person(s) in the organisation operating the FSTD responsible for developing

and maintaining the ESL;

(b)

the method(s) for verification and validation of the entries in the ESL to comply with the

requirements in point ORA.FSTD.120;

(c)

the internal process for modifications in the ESL, including the configuration control

management of the organisation;

(d)

the notification process to the competent authorities in the event of a major modification which

affects the ESL in accordance with point ORA.FSTD.110;

(e)

how the organisation operating the FSTD ensures that the ESL is displayed and accessible for all

FSTD users and authorities as referred to in point ORA.FSTD.115;

(f)

documentation and record-keeping of the ESL in accordance with point ORA.FSTD.240.

CORRECTIVE-ACTION PLAN AND IMPLEMENTATION

(a)

The corrective action plan should address the items raised by the competent authority following

an evaluation, and should include:

(1)

the correction of the items;

(2)

the root cause analysis, in cases it is applicable;

(3)

corrective and preventive action(s); and

(4)

the time schedule to implement the actions.

(b)

Depending on the items raised, the organisation may need to take immediate corrective action.

(c)

The corrective action plan should be signed by the person nominated by the organisation

operating the FSTD in accordance with point ORA.GEN.210(b) or his or her delegate.

FSTD PERFORMANCE METRICS

(a)

The organisation operating the FSTD should systematically collect data to develop performance

metrics and perform data analysis in order to do all the following:

(1)

demonstrate the performance and use of each FSTD it operates, considering the number

of the FSTDs and how an FSTD performs within the fleet, as applicable;

(2)

assess how well an FSTD supports its intended use;

(3)

identify recurring, systemic technical or operational issues, based on the results from the

performance metrics for the preceding year(s); and

(4)

take timely corrective or preventive actions as necessary.

(b)

At least once per calendar year and in combination with the dossier as referred to in point (d)

of GM3 ORA.FSTD.100, the organisation operating the FSTD should, for each FSTD it operates,

provide the following performance metrics for the preceding year to the competent authority:

(1)

‘Planned available time’ defined as time, in hours, during which an FSTD is available to be

used for training, testing and checking;

(2)

‘Scheduled training time’ defined as time, in hours, during which an FSTD is scheduled to

deliver training, testing and checking;

(3)

‘FSTD support time’ defined as time, in hours, during which an FSTD is intentionally not

available for training, testing and checking;

(4)

‘FSTD downtime’ defined as time, in hours, during which an FSTD is unavailable,

regardless of the cause and contains:

(i)

FSTD failure time, defined as time lost, in hours, during planned available time due

to FSTD-specific failures;

(ii)

external failure time, defined as time lost, in hours, during planned available time

due to external factors;

(5)

‘Lost training time’ defined as the amount of scheduled training time, in hours, not

delivered due to FSTD downtime;

(6)

‘Number of FSTD discrepancies’ determined as deviations from the expected behaviour

or performance of the FSTD;

(7)

‘Number of interruptions during scheduled training time’ which is defined as unplanned

events that temporarily suspend or disrupt an FSTD session, regardless of the completion

of the session;

(8)

‘FSTD availability’ calculated as follows:

(Planned available time - FSTD downtime)/(Planned available time) * 100; and

(9)

‘FSTD reliability’ calculated as follows:

(Planned available time - FSTD failure Time)/(Planned available time) * 100

(c)

The information related to the metrics should be available month by month and overall, for the

preceding 12-month period.

(d)

The organisation operating the FSTD should carry out an analysis of trends, based on the results

from the performance metrics for at least the preceding 12-month period. Such information

should be provided to the competent authority as part of the performance metrics to support

the oversight process.

(e)

The organisation operating the FSTD may develop and provide additional information or metrics

to the competent authority, as appropriate, to demonstrate the performance and utilisation of

the FSTD.

COMPLIANCE MONITORING PROGRAMME – ORGANISATIONs OPERATING FSTDs

One acceptable means of measuring FSTD performance is contained in ARINC report 433-1 (December

14th, 2007 or as amended) Standard Measurements for Flight Simulation Quality.

devices, in a structured way, to ensure that they remain in compliance with the technical

standards of CS-FSTD(A) and CS-FSTD(H) the applicable qualification basis and continue to be

effective training tools. An effective CM function is also essential to support any level of

extended recurrent evaluation period as permitted by ORA.FSTD.225(b).

The For routine fly-outs of the device, are a specialised part of the audit programme. Iit is

They would be expected to be type rating instructor/examiner (TRI/TRE) qualified on the type,

variant or a group of aircraft, as applicable, and should have experience of simulator evaluations

particularly for the an independent organisation operating the FSTDs not directly associated

with an airline or an approved training organisation. It is vital for such the organisations to

ensure that their users are aware of the importance of the fly-outs as part of the continued

(z)

ARINC 433 provides good guidance on FSTD compliance measurement. Metrics should monitor

not only individual FSTD performance but, for larger organisations, how each FSTD is performing

within the fleet. It is also recommended that metrics data be shared, regularly, with the FSTD

manufacturers to allow monitoring for generic problems such as design issues, which may be

best addressed with a fleet-wide solution.

GENERAL

(a)

Preventive action is the action to eliminate the cause of a potential non-compliance or other

undesirable potential situation and prevent recurrence of an existing detected non-compliance

or other undesirable condition or situation.

(b)

Corrective action is the action to eliminate or mitigate the root cause(s) of a detected non-

compliance.

(c)

Correction is the action to eliminate a detected non-compliance.

ROOT CAUSE ANALYSIS

(d)

The organisation operating the FSTD may perform a root cause analysis or use another

appropriate tool to identify the root cause of the non-compliance. Proper determination of the

root cause is crucial for defining effective corrective actions to prevent reoccurrence of the non-

compliance on the FSTD.

(e)

It is important that the analysis does not primarily focus on establishing who or what caused

the non-compliance, but on why it was caused. Establishing the root cause(s) of a non-

compliance often requires an overarching view of the circumstances that led to it, to identify all

possible systematic and contributing factors (human factors, regulatory, organisational,

technical factors, etc.) in addition to the direct factors.

(f)

A narrow focus on single events or failures, or the use of a simple, linear model, such as a fault

tree, to identify the chain of events that led to the non-compliance, may not properly reflect

the complexity of the non-compliance, and therefore, there is a risk that important factors that

must be considered to prevent reoccurrence will be ignored. Such an insufficient or partial root

cause analysis often leads to applying ‘quick fixes’ that only address the symptoms of the non-

compliance. A peer review of the results of the root cause analysis may increase its reliability

and objectivity.

(g)

The analysis should consider whether the non-compliance equally affects other FSTDs operated

by the organisation.

FSTD PERFORMANCE METRICS

(a)

Organisations should develop and maintain a consistent methodology to calculate and report

performance metrics. These values are not only a means to demonstrate compliance with point

ORA.FSTD.100(d) but also form part of the organisation’s management system for ensuring

continued FSTD performance. This data can also inform proactive maintenance decisions and

support dialogue with FSTD users, FSTD providers and competent authorities.

(b)

The performance metrics may be used by the organisation operating the FSTD to demonstrate

to the competent authority:

(1)

the performance of an FSTD to facilitate the competent authority’s assessment on

extension of the recurrent evaluation period of the FSTD;

(2)

the effectiveness of its management system with regard to the FSTD operation and

maintenance.

(c)

For calculation of the metrics as referred to in point (b) of AMC1 ORA.FSTD.100(d), the following

may serve as guidance:

(1)

‘FSTD support time’ may include the following times:

(i)

out of service: closure of the training centre, scheduled out of service because of

major update;

(ii)

maintenance: preventative and corrective;

(iii)

engineering: development, improvement of the FSTD;

(iv)

regulatory: authority evaluation, QTG rerun, activities under point ORA.FSTD.225,

fly-out;

(v)

configuration: change of configuration, time between two FSTD sessions, etc.

(2)

‘FSTD downtime’ can be during scheduled training or even when the training is not

impacted. This metric takes into account all events that could affect the availability of the

FSTD.

(3)

‘External failure time’ is time lost during planned available time due to external factors

such as facility related causes; operator/user causes; scheduling/logistical causes;

environmental or external disruptions.

COMPLIANCE MONITORING MANAGEMENT SYSTEM – GUIDANCE FOR ORGANISATIONS

OPERATING FSTDs WHEN TO PREPARINGE FOR A COMPETENT AUTHORITY EVALUATION

support the discussion during the preliminary briefing, which is a the first step of any initial or

This document material has been developed as well to standardise working methods

At least seven days prior to the date of the planned evaluation, the organisation operating the

FSTD should provide the competent authority with the dossiers for initial and recurrent

evaluation as referred to in points (c) and (d).

FSTD capability signature (FCS) type of FSTD and qualification level requested;

evaluation agenda: including date of the evaluation, name of the people involved for the

computers, if applicable. Manuals Test documentation needed for an evaluation (e.g.

type of FSTD and qualification level or FCS or assigned FCS, as applicable requested;

evaluation agenda, including date of the evaluation, name of the people involved for the

reliability data: training hours month by month during the past year, numbers of

complaints mentioned in the technical log, training hours lost, availability rate FSTD

metrics to demonstrate the FSTD performance, use, and other characteristics as referred

to in point ORA.FSTD.100(d);

hardware and/or software updates or changes since the last evaluation and planned

(10) subjective open defect(s) and a list of all reported defects since the last evaluation or, in

cases it is applicable, since the activities performed in accordance with point

ORA.FSTD.225(b);

past year, any comment, and the status of the tests; and

evaluation and a summary of actions taken;.

(14) current ESL revision number and revision date; and

(15) current software load reference.