Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
Name
3715275.3732123.pdf
Size
2.82 MB
Format
Adobe PDF
Checksum (MD5)
e0b54e48841f01ff368726ad73d4c86d
Author(s)
Gosciak, Jennah
Balagopalan, Aparna
Ouyang, Derek
Koenecke, Allison
Ghassemi, Marzyeh
Ho, Daniel
Date Issued
June 23, 2025
Publisher
ACM|The 2025 ACM Conference on Fairness, Accountability, and Transparency
Citation
Jennah Gosciak, Aparna Balagopalan, Derek Ouyang, Allison Koenecke, Marzyeh Ghassemi, and Daniel E. Ho. 2025. Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT '25). Association for Computing Machinery, New York, NY, USA, 1843–1861.
Version
Final published version
Abstract
Prior work has documented widespread racial and ethnic inequities across sectors, such as healthcare, finance, and technology. Across all of these domains, conducting disparity assessments at regular time intervals is critical for surfacing potential biases in decision-making and improving outcomes across demographic groups. Because disparity assessments fundamentally depend on the availability of demographic information, their efficacy is limited by the availability and consistency of available demographic identifiers. While prior work has considered the impact of missing data on fairness, little attention has been paid to the role of delayed demographic data. Delayed data, while eventually observed, might be missing at the critical point of monitoring and action – and delays may be unequally distributed across groups in ways that distort disparity assessments. We characterize such impacts in healthcare, using electronic health records of over 5M patients across primary care practices in all 50 states. Our contributions are threefold. First, we document the high rate of race and ethnicity reporting delays in a healthcare setting and demonstrate widespread variation in rates at which demographics are reported across different groups. Second, through a set of retrospective analyses using real data, we find that such delays impact disparity assessments and hence conclusions made across a range of consequential healthcare outcomes, particularly at more granular levels of state-level and practice-level assessments. Third, we find limited ability of conventional methods that impute missing race in mitigating the effects of reporting delays on the accuracy of timely disparity assessments. Our insights and methods generalize to many domains of algorithmic fairness where delays in the availability of sensitive information may confound audits, thus deserving closer attention within a pipeline-aware machine learning framework.
Description
FAccT ’25, Athens, Greece
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3715275.3732123
