COGNIMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization
Name
13640_2017_Article_194.pdf
Size
2.67 MB
Format
Adobe PDF
Checksum (MD5)
b27d5104a0907515cc2c900ebfaf2943
Author(s)
Zlatintsi, Athanasia
Koutras, Petros
Malandrakis, Nikolaos
Efthymiou, Niki
Pastra, Katerina
Potamianos, Alexandros
Maragos, Petros
Evangelopoulos, Georgios
Date Issued
August 2017
Journal
EURASIP Journal on Image and Video Processing
Publisher
Springer International Publishing
Citation
Zlatintsi, Athanasia, et al. “COGNIMUSE: A Multimodal Video Database Annotated with Saliency, Events, Semantics and Emotion with Application to Summarization.” EURASIP Journal on Image and Video Processing, vol. 2017, no. 1, Dec. 2017.
Version
Final published version
Abstract
Abstract: Research related to computational modeling for machine-based understanding requires ground truth data for training, content analysis, and evaluation. In this paper, we present a multimodal video database, namely COGNIMUSE, annotated with sensory and semantic saliency, events, cross-media semantics, and emotion. The purpose of this database is manifold; it can be used for training and evaluation of event detection and summarization algorithms, for classification and recognition of audio-visual and cross-media events, as well as for emotion tracking. In order to enable comparisons with other computational models, we propose state-of-the-art algorithms, specifically a unified energy-based audio-visual framework and a method for text saliency computation, for the detection of perceptually salient events from videos. Additionally, a movie summarization system for the automatic production of summaries is presented. Two kinds of evaluation were performed, an objective based on the saliency annotation of the database and an extensive qualitative human evaluation of the automatically produced summaries, where we investigated what composes high-quality movie summaries, where both methods verified the appropriateness of the proposed methods. The annotation of the database and the code for the summarization system can be found at http://cognimuse.cs.ntua.gr/database
Keywords: Video database, SaliencyCross-media relations, Emotion annotation, Audio-visual events, Video summarization .
MIT Department
McGovern Institute for Brain Research at MIT
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1186/s13640-017-0194-1
