Improving the efficiency of research and development using belief networks

Abstract

Within the past thirty years, the U.S. government has spent over three trillion dollars supporting research and development projects across its various federal agencies. There is a considerable, long-standing need to monitor, justify, and improve this allocation of taxpayer monies. However, oversight and prioritization of research funding has been haphazard, largely because the agencies that administer research funding lack appropriate metrics to measure project success. We investigate the use of Bayesian belief networks as a means of tracking the success of research and development projects and prioritizing research funding across different experimental efforts. The focus of the thesis is on demonstrating a proof of concept of Bayesian networks by applying the methodology to an alloy research project led by Dr. Ronald Ballinger at the Massachusetts Institute of Technology. We determine the main parameters of interest in the project, establish a network of conditional probability to relate experimental results to alloy viability, perform a Bayesian updating of the research success probability using experimental results, and examine how the optimal choice of experimental design changes as new information is obtained. We find that belief networks are an appropriate tool for tracking and improving upon the efficiency of research and development. Some potential hurdles are discussed: researcher overconfidence, computational limits of Monte Carlo assessment, and principal-agent games. We reach the conclusion that belief networks are applicable to research and development projects and that their use should be endorsed by the Office of Management and Budget as a means of improving accountability among research-intensive federal agencies.