Ex Ante Evaluation and Improvement of Forecasts

Abstract

The dominant approach reported in the literature is to evaluate forecasts after the fact. We take a different approach, we present a way to evaluate and Improve forecasts before the fact. We reconceptualize forecasts as thought experiments grounded on mental models. We show the results of our process which debiases and reduces the asymmetry of forecasters’ mental models. We also reconceptualize forecasting as measurements with errors. And to analyze and improve the entire forecasting process as a system, we use the methods of Design of Experiments (DOE) and Gage R&R from Measurement System Analysis (MSA). We show the results of our analyses using two new metrics, repeatability and reproducibility and discuss new opportunities for research.