Limitations of Commercial Aviation Safety Assessment Standards Uncovered in the Wake of the Boeing 737 MAX Accidents

Abstract

Commercial aviation accidents, though exceedingly rare, come at a large human, economic, and social cost. Therefore, different stakeholders in industry and government have collaborated to develop standard processes for developing aircraft and assessing their safety, the most popular being the Society of Automotive Engineers’ (SAE) Aerospace Recommended Practices (ARPs) 4754 and 4761. However, most of the engineering techniques used for aircraft development and safety assessment were developed in the mid-20th century and formalized into these standards in the 1990s. Modern aircraft often involve complex interactions between hardware, software, and humans, and the engineering techniques used to analyze these systems have not kept up with the pace of technological development. This thesis studies two recent accidents involving the Boeing 737 MAX (Lion Air flight JT610 and Ethiopian Airlines flight ET302) to identify the limitations that still exist in aviation safety assessment guidance that have contributed to these accidents. A new accident analysis methodology called Causal Analysis based on Systems Theory (CAST) was applied to the 737 MAX accidents to understand why the complex interactions leading to the accidents were not identified during the safety assessment process. The analysis uncovered four main limitations in safety assessment guidance that contributed to the accidents: (a) limited integration of human factors and safety, (b) limited guidance for identifying assumptions, (c) limited ability to capture non-failure based causal scenarios, and (d) limited ability to understand complex nonlinear causal relationships. A new hazard analysis tool called System-Theoretic Process Analysis (STPA) was then applied to the same systems involved in the 737 MAX accidents to evaluate whether STPA can be used to address the identified limitations. STPA’s scenario-based framework that incorporates humans and software into the hazard analysis was found to support validation of human response assumptions, identification of new assumptions, assessing safety of intended behavior, and understanding circular causality or otherwise non-linear causal factors.