Operational Considerations in the Certification of Technically Advanced

Operational Considerations in the Certification of Technically Advanced Aircraft (TAA) Robert Barnes, Robert B Barnes Associates, Inc. (USA) Charles Adam, Federal Aviation Administration (USA) Operational Considerations in the Certification of TAA

Today’s topics • The need to consider the user • What we’ve seen during flight test • Current policy • Current design practice • Importance of real user trials • Our suggestions Operational Considerations in the Certification of TAA

“General aviation today is experiencing the most rapid pace of innovation since the late 1940 s … … These advances offer general aviation pilots the promise of increased levels of safety and performance … … Unfortunately, these increased levels of safety have not materialized. ” -- Littlefield, Glass Cockpit Flying, 2010 Operational Considerations in the Certification of TAA

“Why has the promise of greater levels of safety for glass cockpit airplanes not been realized? Because, in order to realize these benefits, general aviation pilots must learn a new way of flying. Unfortunately, general aviation pilots and training providers have not yet evolved the way they train and fly to catch up with the advances in glass cockpit technology. ” -- Linked. In Aviation Training Group Discussion Operational Considerations in the Certification of TAA

As seen during flight test • Design may be functionally acceptable to engineers • Is the design acceptable to properly qualified pilots? • Can a crew safely operate new design? Operational Considerations in the Certification of TAA

Some personal findings • Engineers tend to design systems which may seem logical and intuitive to them (an engineering skill) – Sequences of steps – Multifunction control logic • These same systems may be far from logical or intuitive to the typical pilot (a pilot skill) – Recognizing operation of dedicated controls Operational Considerations in the Certification of TAA

Effectively demonstrating design acceptability requires: Involvement of qualified pilots … Representing a cross-section of the target pilot population … Who are operating the new design under realistic conditions. Operational Considerations in the Certification of TAA

Human Factors in Part 23 Aircraft Certification • FAA Certification Policy on Human Factors (PS-ACE 1002001 -004, Aug 2002) - Response to industry requests for guidance - Advocates early FAA involvement during development - Goal: To aid in the timely identification and resolution of human factors-related issues during the aircraft certification process - Guidance only … not a regulatory requirement • “General Aviation fatalities (part 91 operations) occur at higher rates than part 121 and part 135 operations. ” Operational Considerations in the Certification of TAA

Operational Considerations • Pilot characteristics • Operational considerations • System operation procedures • Training requirements Operational Considerations in the Certification of TAA

FAA Policy on Human Factors • Not a regulatory requirement but FAA ACO “should attempt” to follow • Increased effort being made to address designrelated human error • FAA rulemaking effort (25. 1302) already being applied by EASA • Human factors issues are open to broad interpretation – Component level – Operational level Operational Considerations in the Certification of TAA

Human Factors Design Practice • Design and Operational factors affect workload • There are many ways to evaluate these affects • 500 pages of methods described • User trials are on page 475 Operational Considerations in the Certification of TAA

Some user trial advantages • Simplistic and flexible approach to usability evaluation • Evaluates the system based upon the potential end users’ performance • Gives the designers a powerful insight into how the system will be used Operational Considerations in the Certification of TAA

Some user trial disadvantages • Time consuming to conduct • Large amounts of data are collected, ensuring a lengthy analysis process • It may be difficult to gain access to the required personnel or end users • End users may be biased towards the old system or procedure Operational Considerations in the Certification of TAA

Applying User Trials in U. S. • Limited form now used periodically by FAA for avionic component evaluations • Multiple Pilot System Usability Evaluation (MPSUE) • FAA pilots not typical end users Does this adequately evaluate operation by the typical user? Operational Considerations in the Certification of TAA

TAA certification user trial • Removing restrictions required a human factors workload evaluation • Applicant chose to use Modified Pilot Subjective Evaluation (MPSE) • Two typical user pilots were qualified • Certification was successful but many lessons were learned by manufacturer, training provider, and regulator Operational Considerations in the Certification of TAA

Conclusions • Human factors evaluation is subjective • Level of pilot competence required to operate design is critical • Using experienced test pilots is not enough “Is the aircraft capable of being operated safely by the manufacturer’s target customer with the type of training proposed? ” Operational Considerations in the Certification of TAA

Questions/Comments? For a copy of our papers, please leave me your card Operational Considerations in the Certification of TAA