Crowd equals diversity? a diversity analysis on participation of agency-sponsored open innovation challenges

Abstract

Recent events in the US and beyond have stirred societal debates on how to measure and increase diversity in fields such as Science, Technology, Engineering, and Math (STEM). In the past decade, many US public agencies with STEM missions have explored crowd-sourced, open innovation challenges as a potential solution. To probe whether and how diversity has been achieved in this context, this thesis addresses three goals: (i) understanding the multi-faceted nature of diversity and its determinants based on literature review across disciplines, (ii) testing whether open innovation challenges achieve the policy objective of increasing diversity in solvers for public agencies, and (iii) investigating whether increased diversity leads to desired innovation outcomes. This thesis leverages the NASA-Freelancer Astrobee Challenge Series (ACS), an online robotics design prize competition, as a case study.

Through a systematic literature review, this thesis proposes a multi-dimensional diversity definition, covering surface vs. deep, absolute vs. relative, and individual vs. population diversity perspectives. It adopts a mixed approach of inductive coding and natural language processing to exact motivation from solvers; one-way ANOVA and Tukey test to examine the significance between diversity and solution quality generated in ACS. The study finds that ACS, a representative of open innovation challenges, does not generate all dimensions of diversity in its solvers. On the surface-level dimension of diversity, we observe that ACS achieves greater inclusion in some features such as age and country, but not so much on gender. ACS also achieves some extents of deep-level diversity, reflected in features such as education, career backgrounds, and motivation.

Furthermore, insights from the study show that achieving diversity of solvers in certain features does not necessarily lead to quality solutions. Based on these insights, this thesis formulates a series of policy implications for STEM federal agencies, exploring the effectiveness of open innovation challenges as a policy mechanism for increasing diversity. It also generates suggestions to improve solution quality of future open innovation challenges.