2026-09-20T04:59:46Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1648312026-02-13T03:49:08Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Andrews, Isaiah Siegmann, Charlotte B. Massachusetts Institute of Technology. Department of Economics 2026-02-12T17:13:25Z 2026-02-12T17:13:25Z 2025-09 2025-09-15T14:57:43.945Z https://hdl.handle.net/1721.1/164831 Double ML estimators can estimate coefficients of interest with far fewer functional form assumptions than linear econometric methods. However, DML requires researchers to make a range of implementation choices, including the selection of the function class, the random seed, and hyperparameter configurations. While asymptotic theory suggests these choices should not affect final estimates, we show that for 10 economic analyses (8 of them published and peer-reviewed), implementation choices affect the results. In half of the datasets, different implementation choices even change the interpretation of findings between negative, null, or positive effects. We link these results to a framework for empirically assessing the performance of machine-learning-based estimators, focusing on precision, coverage, and susceptibility to manipulation. This is meant to complement asymptotic theory. We demonstrate that the coverage of DML confidence intervals is too low—placing an upper bound of 48% on the expected coverage of conventional 95% confidence intervals for published DML economics papers. We show that in the status quo, the susceptibility of DML to manipulation by researchers is high, but propose ways to mitigate this susceptibility. S.M. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ Under-Coverage of Double Machine Learning Due to Implementation Choices Thesis application/pdf Master Master of Science in Economics Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="cc5089bb-fb52-42e7-bdb0-33b05dc68f82"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Under-Coverage of Double Machine Learning Due to Implementation Choices</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-09</PublicationDate> <Authors> <Author> <DisplayName>Siegmann, Charlotte B.</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>Double ML estimators can estimate coefficients of interest with far fewer functional form assumptions than linear econometric methods. However, DML requires researchers to make a range of implementation choices, including the selection of the function class, the random seed, and hyperparameter configurations. While asymptotic theory suggests these choices should not affect final estimates, we show that for 10 economic analyses (8 of them published and peer-reviewed), implementation choices affect the results. In half of the datasets, different implementation choices even change the interpretation of findings between negative, null, or positive effects. We link these results to a framework for empirically assessing the performance of machine-learning-based estimators, focusing on precision, coverage, and susceptibility to manipulation. This is meant to complement asymptotic theory. We demonstrate that the coverage of DML confidence intervals is too low—placing an upper bound of 48% on the expected coverage of conventional 95% confidence intervals for published DML economics papers. We show that in the status quo, the susceptibility of DML to manipulation by researchers is high, but propose ways to mitigate this susceptibility.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>