Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching

Sarah Ibrahimi, Shuo Chen, Devanshu Arya, Arthur Câmara, Yunlu Chen, Tanja Crijns, Maurits van der Goes, Thomas Mensink, Emiel van Miltenburg, Daan Odijk, William Thong, Jiaojiao Zhao, Pascal Mettes

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.
Original languageEnglish
Title of host publicationProceedings of the 27th ACM International Conference on Multimedia
Place of PublicationNew York, NY, USA
PublisherACM
Pages2196-2198
Number of pages3
ISBN (Print)9781450368896
DOIs
Publication statusPublished - 2019

Publication series

NameMM '19
PublisherACM

Fingerprint

Semantics
Personnel

Keywords

  • exploration
  • matching
  • multimodal
  • semantics
  • video

Cite this

Ibrahimi, S., Chen, S., Arya, D., Câmara, A., Chen, Y., Crijns, T., ... Mettes, P. (2019). Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 2196-2198). (MM '19). New York, NY, USA: ACM. https://doi.org/10.1145/3343031.3350597
Ibrahimi, Sarah ; Chen, Shuo ; Arya, Devanshu ; Câmara, Arthur ; Chen, Yunlu ; Crijns, Tanja ; van der Goes, Maurits ; Mensink, Thomas ; van Miltenburg, Emiel ; Odijk, Daan ; Thong, William ; Zhao, Jiaojiao ; Mettes, Pascal. / Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. Proceedings of the 27th ACM International Conference on Multimedia. New York, NY, USA : ACM, 2019. pp. 2196-2198 (MM '19).
@inproceedings{e0b71f42b7474c34bacd91076a6b3ada,
title = "Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching",
abstract = "This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.",
keywords = "exploration, matching, multimodal, semantics, video",
author = "Sarah Ibrahimi and Shuo Chen and Devanshu Arya and Arthur C{\^a}mara and Yunlu Chen and Tanja Crijns and {van der Goes}, Maurits and Thomas Mensink and {van Miltenburg}, Emiel and Daan Odijk and William Thong and Jiaojiao Zhao and Pascal Mettes",
year = "2019",
doi = "10.1145/3343031.3350597",
language = "English",
isbn = "9781450368896",
series = "MM '19",
publisher = "ACM",
pages = "2196--2198",
booktitle = "Proceedings of the 27th ACM International Conference on Multimedia",

}

Ibrahimi, S, Chen, S, Arya, D, Câmara, A, Chen, Y, Crijns, T, van der Goes, M, Mensink, T, van Miltenburg, E, Odijk, D, Thong, W, Zhao, J & Mettes, P 2019, Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. in Proceedings of the 27th ACM International Conference on Multimedia. MM '19, ACM, New York, NY, USA, pp. 2196-2198. https://doi.org/10.1145/3343031.3350597

Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. / Ibrahimi, Sarah; Chen, Shuo; Arya, Devanshu; Câmara, Arthur; Chen, Yunlu; Crijns, Tanja; van der Goes, Maurits; Mensink, Thomas; van Miltenburg, Emiel; Odijk, Daan; Thong, William; Zhao, Jiaojiao; Mettes, Pascal.

Proceedings of the 27th ACM International Conference on Multimedia. New York, NY, USA : ACM, 2019. p. 2196-2198 (MM '19).

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

TY - GEN

T1 - Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching

AU - Ibrahimi, Sarah

AU - Chen, Shuo

AU - Arya, Devanshu

AU - Câmara, Arthur

AU - Chen, Yunlu

AU - Crijns, Tanja

AU - van der Goes, Maurits

AU - Mensink, Thomas

AU - van Miltenburg, Emiel

AU - Odijk, Daan

AU - Thong, William

AU - Zhao, Jiaojiao

AU - Mettes, Pascal

PY - 2019

Y1 - 2019

N2 - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

AB - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

KW - exploration

KW - matching

KW - multimodal

KW - semantics

KW - video

U2 - 10.1145/3343031.3350597

DO - 10.1145/3343031.3350597

M3 - Conference contribution

SN - 9781450368896

T3 - MM '19

SP - 2196

EP - 2198

BT - Proceedings of the 27th ACM International Conference on Multimedia

PB - ACM

CY - New York, NY, USA

ER -

Ibrahimi S, Chen S, Arya D, Câmara A, Chen Y, Crijns T et al. Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. In Proceedings of the 27th ACM International Conference on Multimedia. New York, NY, USA: ACM. 2019. p. 2196-2198. (MM '19). https://doi.org/10.1145/3343031.3350597