Conducting Non-Adaptive Experiments in a Live Setting: A Bayesian Approach to Determining Optimal Sample Size
Name
v007t06a015-detc2019-98335.pdf
Description
Published version
Size
1.47 MB
Format
Adobe PDF
Checksum (MD5)
2e569069ef3a37011f082fe2304c4ac6
Author(s)
Sudarsanam, Nandan
Chandran, Ramya
Frey, Daniel D.
Date Issued
August 2019
Journal
Proceedings of the ASME Design Engineering Technical Conference
Publisher
ASME International
Citation
Sudarsanam, Nandan, Chandran, Ramya and Frey, Daniel D. 2019. "Conducting Non-Adaptive Experiments in a Live Setting: A Bayesian Approach to Determining Optimal Sample Size." Proceedings of the ASME Design Engineering Technical Conference, 7.
Version
Final published version
Abstract
Copyright © 2019 ASME. This research studies the use of predetermined experimental plans in a live setting with a finite implementation horizon. In this context, we seek to determine the optimal experimental budget in different environments using a Bayesian framework. We derive theoretical results on the optimal allocation of resources to treatments with the objective of minimizing cumulative regret, a metric commonly used in online statistical learning. Our base case studies a setting with two treatments assuming Gaussian priors for the treatment means and noise distributions. We extend our study through analytical and semi-analytical techniques which explore worst-case bounds and the generalization to k treatments. We determine theoretical limits for the experimental budget across all possible scenarios. The optimal level of experimentation that is recommended by this study varies extensively and depends on the experimental environment as well as the number of available units. This highlights the importance of such an approach which incorporates these factors to determine the budget.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1115/detc2019-98335
