Speech, signal, symptom : machine listening and the remaking of psychiatric assessment

Abstract

This multi-sited, ethnographic dissertation follows teams of psychiatric and engineering professionals collaborating to tackle one of Western psychiatry's longest standing issues: the subjective nature of mental illness. Situated at three different U.S.-based universities, the teams are driven by a conviction that conventional methods of psychiatric screening are fallible if not altogether inaccurate, since they depend upon a mental health care worker's ability to interpret the semantic content of a patient's speech. Through research studies involving human subjects, the teams hope to develop more biologically based and resource-efficient screening techniques that instead analyze paralinguistic, acoustic components of speech-such as pitch, speaking rate, and breathiness-which they argue are more directly linked to the internal mechanisms that drive mental illness.

By turning to the expertise of computer scientists and engineers, they seek to build "machine listening" prototypes for psychiatric assessment: technologies that use a microphone to capture sound and artificial intelligence (AI) to analyze sound. While their studies are premised on the notion that AI can listen beyond the human by attending to sounds of speech that have psychopathological significance supposedly set aside from linguistic meaning and human difference, in order to gather and classify the data necessary for building their technologies, researchers must rely on the very components of language that they seek to overcome: its interactional, sociocultural dimensions. I show how the connections between spoken utterances and inner states that researchers design their systems to make "autonomously" depend upon a tightly managed but oftentimes hidden infrastructure of human labor, including the labor of research subjects.

The division of labor within the teams replicates hierarchies of value within mental health care professions, which place diagnosis and treatment at the top as expert, biomedically and legally ratified forms of judgment, and place the data entry and triage work of assessment at the bottom, as skilless, para-professional, and mechanized tasks. In describing the vexed status and ethics of listening, language, labor, and care in contemporary U.S. mental health care, the dissertation tells a larger story about the stakes of framing mental illness as a scientific, bureaucratic problem calling for a technological intervention