Evidence-based AI Ethics

Abstract

With the rise in prominence of algorithmic-decision making, and numerous high-profile failures, many people have called for the integration of ethics into the development and use of these technologies. In the past five years, the field of “AI Ethics” has risen to prominence to explore questions such as 'how can ML algorithms be more fair' and 'are are tradeoffs when incorporating values such as fairness or privacy into models.' One common trend, particularly by corporations and governments, has been a top-down, principles-based approach for setting the agenda. However, such efforts are usually too abstract to engage with; everyone agrees models should be fair, but there is often disagreement on what "fair" means. In this work, I propose a bottom-up alternative: Evidence-based AI Ethics. Learning from other influential movements, such as Evidence-based Medicine, we can consider specific projects and examine them for "evidence." We draw from complementary critical lenses, one based on utilitarian ethics and on from intersectional feminism to analyze five case studies I have worked on, ranging from automatically-generated radiology reports to tech worker organizing.