Coherent approximation of distributed expert assessments

Abstract

Expert judgments of probability and expectation play an integral role in many systems. Financial markets, public policy, medical diagnostics and more rely on the ability of informed experts (both human and machine) to make educated assessments of the likelihood of various outcomes. Experts however are not immune to errors in judgment (due to bias, quantization effects, finite information or many other factors). One way to compensate for errors in individual judgments is to elicit estimates from multiple experts and then fuse the estimates together. If the experts act sufficiently independently to form their assessments, it is reasonable to assume that individual errors in judgment can be negated by pooling the experts' opinions. Determining when experts' opinions are in error is not always a simple matter. However, one common way in which experts' opinions may be seen to be in error is through inconsistency with the known underlying structure of the space of events. Not only is structure useful in identifying expert error, it should also be taken into account when designing algorithms to approximate or fuse conflicting expert assessments. This thesis generalizes previously proposed constrained optimization methods for fusing expert assessments of uncertain events and quantities. The major development consists of a set of information geometric tools for reconciling assessments that are inconsistent with the assumed structure of the space of events. This work was sponsored by the U.S. Air Force under Air Force Contract FA8721- 05-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the United States Government.