Uncertainty assessment of complex models with application to aviation environmental systems

Abstract

Numerical simulation models that support decision-making and policy-making processes are often complex, involving many disciplines, and long computation times. These models typically have many factors of different character, such as operational, design-based, technological, and economics-based. Such factors generally contain uncertainty, which leads to uncertainty in model outputs. For such models, it is critical to both the application of model results and the future development of the model that uncertainty be properly assessed. This thesis presents a comprehensive approach to the uncertainty assessment of complex models intended to support decision- and policy-making processes. The approach consists of seven steps, which are establishing assessment goals, documenting assumptions and limitations, documenting model factors and outputs, classifying and characterizing factor uncertainty, conducting uncertainty analysis, conducting sensitivity analysis, and presenting results. Factor uncertainty is represented probabilistically, characterized by the principle of maximum uncertainty, and propagated via Monte Carlo simulation. State-of-the-art methods of global sensitivity analysis are employed to apportion model output variance across model factors, and a fundamental extension of global sensitivity analysis, termed distributional sensitivity analysis, is developed to determine on which factors future research should focus to reduce output variability.

(cont.) The complete approach is demonstrated on a real-world model intended to estimate the impacts of aviation on climate change in support of decision- and policy-making, where it is established that a systematic approach to uncertainty assessment is critical to the proper application and future development of complex models. A novel surrogate modeling methodology designed specifically for uncertainty assessment is also presented and demonstrated for an aircraft emissions prediction model that is being developed and applied to support aviation environmental policy-making. The results demonstrate how confidence intervals on surrogate model predictions can be used to balance the tradeoff between computation time and uncertainty in the estimation of statistical outputs of interest in uncertainty assessment.