MIT Libraries homeMIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

COGNIMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization

Author(s)
Zlatintsi, Athanasia; Koutras, Petros; Malandrakis, Nikolaos; Efthymiou, Niki; Pastra, Katerina; Potamianos, Alexandros; Maragos, Petros; Evangelopoulos, Georgios; ... Show more Show less
Thumbnail
Download13640_2017_Article_194.pdf (2.667Mb)
PUBLISHER_CC

Publisher with Creative Commons License

Creative Commons Attribution

Terms of use
Creative Commons Attribution http://creativecommons.org/licenses/by/4.0/
Metadata
Show full item record
Abstract
Abstract: Research related to computational modeling for machine-based understanding requires ground truth data for training, content analysis, and evaluation. In this paper, we present a multimodal video database, namely COGNIMUSE, annotated with sensory and semantic saliency, events, cross-media semantics, and emotion. The purpose of this database is manifold; it can be used for training and evaluation of event detection and summarization algorithms, for classification and recognition of audio-visual and cross-media events, as well as for emotion tracking. In order to enable comparisons with other computational models, we propose state-of-the-art algorithms, specifically a unified energy-based audio-visual framework and a method for text saliency computation, for the detection of perceptually salient events from videos. Additionally, a movie summarization system for the automatic production of summaries is presented. Two kinds of evaluation were performed, an objective based on the saliency annotation of the database and an extensive qualitative human evaluation of the automatically produced summaries, where we investigated what composes high-quality movie summaries, where both methods verified the appropriateness of the proposed methods. The annotation of the database and the code for the summarization system can be found at http://cognimuse.cs.ntua.gr/database Keywords: Video database, SaliencyCross-media relations, Emotion annotation, Audio-visual events, Video summarization .
Date issued
2017-08
URI
http://hdl.handle.net/1721.1/113872
Department
McGovern Institute for Brain Research at MIT
Journal
EURASIP Journal on Image and Video Processing
Publisher
Springer International Publishing
Citation
Zlatintsi, Athanasia, et al. “COGNIMUSE: A Multimodal Video Database Annotated with Saliency, Events, Semantics and Emotion with Application to Summarization.” EURASIP Journal on Image and Video Processing, vol. 2017, no. 1, Dec. 2017.
Version: Final published version
ISSN
1687-5281
1687-5176

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries homeMIT Libraries logo

Find us on

Twitter Facebook Instagram YouTube RSS

MIT Libraries navigation

SearchHours & locationsBorrow & requestResearch supportAbout us
PrivacyPermissionsAccessibility
MIT
Massachusetts Institute of Technology
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.