Uncertainty and individual discretion in allocating research funds

Abstract

There is a long-standing tradition in public research funding agencies of distributing funds via peer review, which aggregates evaluations of proposed research ideas from a group of external experts. Despite complaints that this process is biased against novel ideas, there is poor understanding of an alternative system that may overcome this bias: the use of individual discretion. Here, we conduct the first quantitative study of how individual discretion affects a research funding portfolio. Using internal project selection data from the Advanced Research Projects Agency-Energy (ARPA-E), we describe how a portfolio of projects selected by individual discretion differs from a portfolio of projects selected by traditional peer review. We show that ARPA-E program directors prefer to fund proposals with greater disagreement among experts, especially if at least one reviewer thinks highly of the proposal. This preference leads ARPA-E to fund more uncertain and creative research ideas, which supports the agency's mission of pursuing novel ideas for transformational energy technology.