Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching

Sarah Ibrahimi; Shuo Chen; Devanshu Arya; Arthur Câmara; Yunlu Chen; Tanja Crijns; Maurits van der Goes; Thomas Mensink; Emiel van Miltenburg; Daan Odijk; William Thong; Jiaojiao Zhao; Pascal Mettes

doi:10.1145/3343031.3350597

Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching

Sarah Ibrahimi, Shuo Chen, Devanshu Arya, Arthur Câmara, Yunlu Chen, Tanja Crijns, Maurits van der Goes, Thomas Mensink, Emiel van Miltenburg, Daan Odijk, William Thong, Jiaojiao Zhao, Pascal Mettes

Language, Communication and Cognition

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Abstract

This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

Original language	English
Title of host publication	Proceedings of the 27th ACM International Conference on Multimedia
Place of Publication	New York, NY, USA
Publisher	ACM
Pages	2196-2198
Number of pages	3
ISBN (Print)	9781450368896
DOIs	https://doi.org/10.1145/3343031.3350597
Publication status	Published - 2019

Publication series

Name	MM '19
Publisher	ACM

Keywords

exploration
matching
multimodal
semantics
video

Access to Document

10.1145/3343031.3350597

http://doi.acm.org/10.1145/3343031.3350597

Fingerprint Dive into the research topics of 'Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching'. Together they form a unique fingerprint.

Cite this

Ibrahimi, S., Chen, S., Arya, D., Câmara, A., Chen, Y., Crijns, T., van der Goes, M., Mensink, T., van Miltenburg, E., Odijk, D., Thong, W., Zhao, J., & Mettes, P. (2019). Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 2196-2198). (MM '19). ACM. https://doi.org/10.1145/3343031.3350597

Ibrahimi, Sarah ; Chen, Shuo ; Arya, Devanshu ; Câmara, Arthur ; Chen, Yunlu ; Crijns, Tanja ; van der Goes, Maurits ; Mensink, Thomas ; van Miltenburg, Emiel ; Odijk, Daan ; Thong, William ; Zhao, Jiaojiao ; Mettes, Pascal. / Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. Proceedings of the 27th ACM International Conference on Multimedia. New York, NY, USA : ACM, 2019. pp. 2196-2198 (MM '19).

@inproceedings{e0b71f42b7474c34bacd91076a6b3ada,

title = "Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching",

abstract = "This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.",

keywords = "exploration, matching, multimodal, semantics, video",

author = "Sarah Ibrahimi and Shuo Chen and Devanshu Arya and Arthur C{\^a}mara and Yunlu Chen and Tanja Crijns and {van der Goes}, Maurits and Thomas Mensink and {van Miltenburg}, Emiel and Daan Odijk and William Thong and Jiaojiao Zhao and Pascal Mettes",

year = "2019",

doi = "10.1145/3343031.3350597",

language = "English",

isbn = "9781450368896",

series = "MM '19",

publisher = "ACM",

pages = "2196--2198",

booktitle = "Proceedings of the 27th ACM International Conference on Multimedia",

}

Ibrahimi, S, Chen, S, Arya, D, Câmara, A, Chen, Y, Crijns, T, van der Goes, M, Mensink, T, van Miltenburg, E, Odijk, D, Thong, W, Zhao, J & Mettes, P 2019, Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. in Proceedings of the 27th ACM International Conference on Multimedia. MM '19, ACM, New York, NY, USA, pp. 2196-2198. https://doi.org/10.1145/3343031.3350597

Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching. / Ibrahimi, Sarah; Chen, Shuo; Arya, Devanshu; Câmara, Arthur; Chen, Yunlu; Crijns, Tanja; van der Goes, Maurits; Mensink, Thomas; van Miltenburg, Emiel; Odijk, Daan; Thong, William; Zhao, Jiaojiao; Mettes, Pascal.

Proceedings of the 27th ACM International Conference on Multimedia. New York, NY, USA : ACM, 2019. p. 2196-2198 (MM '19).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Interactive Exploration of Journalistic Video Footage Through Multimodal Semantic Matching

AU - Ibrahimi, Sarah

AU - Chen, Shuo

AU - Arya, Devanshu

AU - Câmara, Arthur

AU - Chen, Yunlu

AU - Crijns, Tanja

AU - van der Goes, Maurits

AU - Mensink, Thomas

AU - van Miltenburg, Emiel

AU - Odijk, Daan

AU - Thong, William

AU - Zhao, Jiaojiao

AU - Mettes, Pascal

PY - 2019

Y1 - 2019

N2 - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

AB - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

KW - exploration

KW - matching

KW - multimodal

KW - semantics

KW - video

U2 - 10.1145/3343031.3350597

DO - 10.1145/3343031.3350597

M3 - Conference contribution

SN - 9781450368896

T3 - MM '19

SP - 2196

EP - 2198

BT - Proceedings of the 27th ACM International Conference on Multimedia

PB - ACM

CY - New York, NY, USA

ER -