Quantifying Bias in a Face Verification System
Name
csmf-03-00006.pdf
Size
2.67 MB
Format
Adobe PDF
Checksum (MD5)
780f3110c1371a779cf12d8118e73fcc
Author(s)
Frisella, Megan
Khorrami, Pooya
Matterer, Jason
Kratkiewicz, Kendra
Torres-Carrasquillo, Pedro
Date Issued
April 20, 2022
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Computer Sciences & Mathematics Forum 3 (1): 6 (2022)
Version
Final published version
Abstract
Machine learning models perform face verification (FV) for a variety of highly consequential applications, such as biometric authentication, face identification, and surveillance. Many state-of-the-art FV systems suffer from unequal performance across demographic groups, which is commonly overlooked by evaluation measures that do not assess population-specific performance. Deployed systems with bias may result in serious harm against individuals or groups who experience underperformance. We explore several fairness definitions and metrics, attempting to quantify bias in Google’s FaceNet model. In addition to statistical fairness metrics, we analyze clustered face embeddings produced by the FV model. We link well-clustered embeddings (well-defined, dense clusters) for a demographic group to biased model performance against that group. We present the intuition that FV systems underperform on protected demographic groups because they are less sensitive to differences between features within those groups, as evidenced by clustered embeddings. We show how this performance discrepancy results from a combination of representation and aggregation bias.
MIT Department
Lincoln Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/cmsf2022003006
