System for rapid subtitling

Abstract

A system for rapid subtitling of audiovisual sequences was developed, and evaluated. This new system resulted in average time-savings of 50% over the previous work in the field. To subtitle a 27-minute English lecture, users saved 2.2 hours, spending an average of 1.9 hours timing the monologue. Subtitling a 20-minute Japanese animation resulted in similar savings. While subtitling is widely practiced, traditional speech boundary detection algorithms have proven insufficient for adoption when subtitling audiovisual sequences. Human-edited timing of subtitles is a dull and laborious exercise, yet with some human-guidance, basic systems may accurately align transcripts and translations in the presence of divergent speakers, multiple languages, and background noise. Since no known, fully-automatic recognition systems operate with sufficient accuracy for production-grade subtitle times, during playback the system disclosed treats user-supplied times as a priori data and adjusts those times using extracted features from concurrent data streams. In the application's oracle subsystem, packaged algorithms called oracles preprocess, filter, and present media data, using that data to adjust user-supplied times.

(cont.) The on-the-fly timing subsystem gathers user-supplied times and coordinates the oracles' adjustment of the data. The application was developed with attention to user interface design and data format support, and was sent to users for structured feedback. From these data, system performance is discussed in terms of subtitle time accuracy, time spent by users, and avoidance of common pitfalls in subtitle generation.