Reengineering metrics systems for aircraft sustainment teams : a metrics thermostat for use in strategic priority management

Abstract

We explore the selection of metrics for the United States Air Force weapon system sustainment team empirically with emphasis placed on the incentive, structural and predictive implications of metrics. We define the term "metric" to include measures that employees impact through their efforts. We believe that even in a not-for-profit organization such as the Air Force, by putting emphasis (or weight) on a performance metric, the organization establishes inherent incentive structures within which employees will act to maximize their best interests. However, we believe that not-for-profit organizations differ from for-profit ones in their inherent structure since profit becomes cost and several mission-oriented outcome variables share a fundamental importance in achieving the organizations goals. We seek an understanding of the structural composition of Air Force sustainment's metrics systems that, when coupled with a method for practical selection of a high-quality set of metrics (and weights), will align the incentives of employees with the interests of the organization. The empirical study is grounded in emerging theoretical work, which uses our above definition of a metric to purpose a theoretical metrics feedback construct called the Metrics Thermostat. System structure is explored through common correlation and regression analysis as well as more sophisticated structural equation modeling and systems dynamics techniques used to explore potential feedback loops. The F- 16 is used as a case study for this problem, and the metrics systems are considered from the front-line base-level point of view of Air Force active duty, Air National Guard and Air Force Reserve bases worldwide. 96 low-level metrics, covariates and outcomes were examined for 45 F- 16 bases for a period of five years. Outcome importance was determined through personal interviews and internal archival documentation. -- The metrics, covariates and outcomes in the study are very interrelated. -- The primary indicator of overall performance is Command (ACC, USAFE, etc.) -- Increased Fix Rate increases Utilization, but increased Utilization decreases Fix Rate. -- Cannibalization Rate is associated with higher Fix Rates but lower Mission Capability, Flying Scheduling Effectiveness, and Aircraft Utilization. -- Active duty Mission Capability is predicted well from the dataset such that: * Active duty commands have higher mission capability. * Mission Capability is slightly higher in cool moist climates. * Increased Aircraft Utilization, Repeat Discrepancies and Flying Scheduling Effectiveness are all associated with higher Mission Capability. * Increased Break Rates and Unscheduled (engine) Maintenance are associated with lower Mission Capability. The model appears to be valid for peacetime actions only.