Conducting Non-adaptive Experiments in a Live Setting: A Bayesian Approach to Determining Optimal Sample Size
Name
md_142_3_031108.pdf
Description
Published version
Size
661.63 KB
Format
Adobe PDF
Checksum (MD5)
1bda6b3b0bb86b4db25226a8a7c3fd60
Author(s)
Sudarsanam, Nandan
Chandran, Ramya
Frey, Daniel D
Date Issued
2020
Journal
Journal of Mechanical Design
Publisher
ASME International
Version
Final published version
Abstract
© 2019 by ASME. This research studies the use of predetermined experimental plans in a live setting with a finite implementation horizon. In this context, we seek to determine the optimal experimental budget in different environments using a Bayesian framework. We derive theoretical results on the optimal allocation of resources to treatments with the objective of minimizing cumulative regret, a metric commonly used in online statistical learning. Our base case studies a setting with two treatments assuming Gaussian priors for the treatment means and noise distributions. We extend our study through analytical and semi-analytical techniques which explore worst-case bounds, the presence of unequal prior distributions, and the generalization to k treatments. We determine theoretical limits for the experimental budget across all possible scenarios. The optimal level of experimentation that is recommended by this study varies extensively and depends on the experimental environment as well as the number of available units. This highlights the importance of such an approach which incorporates these factors to determine the budget.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1115/1.4045603
