Automatic mashup generation from multiple-camera concert recordings

P. Shrestha, P.H.N. With de, J. Weda, M. Barbieri, E.H.L. Aarts

Research output: Other contributionOther research output

Abstract

A large number of videos are captured and shared by the audience from musical concerts. However, such recordings are typically perceived as boring mainly because of their limited view, poor visual quality and incomplete coverage. It is our objective to enrich the viewing experience of these recordings by exploiting the abundance of content from multiple sources. In this paper, we propose a novel Virtual Director system that automatically combines the most desirable segments from different recordings resulting in a single video stream, called mashup. We start by eliciting requirements from focus groups, interviewing professional video editors and consulting film grammar literature. We design a formal model for automatic mashup generation based on maximizing the degree of fulfillment of the requirements. Various audio-visual content analysis techniques are used to determine how well the requirements are satisfied by a recording. To validate the system, we compare our mashups with two other mashups: manually created by a professional video editor and machine generated by random segment selection. The mashups are evaluated in terms of visual quality, content diversity and pleasantness by 40 subjects. The results show that our mashups and the manual mashups are perceived as comparable, while both of them are significantly higher than the random mashups in all three terms.
Original languageEnglish
PublisherAssociation for Computing Machinery, Inc
Number of pages10
ISBN (Print)978-1-60558-933-6
DOIs
Publication statusPublished - 2010
Externally publishedYes

Fingerprint

Boring
Cameras

Cite this

Shrestha, P., With de, P. H. N., Weda, J., Barbieri, M., & Aarts, E. H. L. (2010). Automatic mashup generation from multiple-camera concert recordings. Association for Computing Machinery, Inc. https://doi.org/10.1145/1873951.1874023
Shrestha, P. ; With de, P.H.N. ; Weda, J. ; Barbieri, M. ; Aarts, E.H.L. / Automatic mashup generation from multiple-camera concert recordings. 2010. Association for Computing Machinery, Inc. 10 p.
@misc{adeddbbcf3c74babb5bbba7df04e5805,
title = "Automatic mashup generation from multiple-camera concert recordings",
abstract = "A large number of videos are captured and shared by the audience from musical concerts. However, such recordings are typically perceived as boring mainly because of their limited view, poor visual quality and incomplete coverage. It is our objective to enrich the viewing experience of these recordings by exploiting the abundance of content from multiple sources. In this paper, we propose a novel Virtual Director system that automatically combines the most desirable segments from different recordings resulting in a single video stream, called mashup. We start by eliciting requirements from focus groups, interviewing professional video editors and consulting film grammar literature. We design a formal model for automatic mashup generation based on maximizing the degree of fulfillment of the requirements. Various audio-visual content analysis techniques are used to determine how well the requirements are satisfied by a recording. To validate the system, we compare our mashups with two other mashups: manually created by a professional video editor and machine generated by random segment selection. The mashups are evaluated in terms of visual quality, content diversity and pleasantness by 40 subjects. The results show that our mashups and the manual mashups are perceived as comparable, while both of them are significantly higher than the random mashups in all three terms.",
author = "P. Shrestha and {With de}, P.H.N. and J. Weda and M. Barbieri and E.H.L. Aarts",
year = "2010",
doi = "10.1145/1873951.1874023",
language = "English",
isbn = "978-1-60558-933-6",
publisher = "Association for Computing Machinery, Inc",
type = "Other",

}

Shrestha, P, With de, PHN, Weda, J, Barbieri, M & Aarts, EHL 2010, Automatic mashup generation from multiple-camera concert recordings. Association for Computing Machinery, Inc. https://doi.org/10.1145/1873951.1874023

Automatic mashup generation from multiple-camera concert recordings. / Shrestha, P.; With de, P.H.N.; Weda, J.; Barbieri, M.; Aarts, E.H.L.

10 p. Association for Computing Machinery, Inc. 2010, .

Research output: Other contributionOther research output

TY - GEN

T1 - Automatic mashup generation from multiple-camera concert recordings

AU - Shrestha, P.

AU - With de, P.H.N.

AU - Weda, J.

AU - Barbieri, M.

AU - Aarts, E.H.L.

PY - 2010

Y1 - 2010

N2 - A large number of videos are captured and shared by the audience from musical concerts. However, such recordings are typically perceived as boring mainly because of their limited view, poor visual quality and incomplete coverage. It is our objective to enrich the viewing experience of these recordings by exploiting the abundance of content from multiple sources. In this paper, we propose a novel Virtual Director system that automatically combines the most desirable segments from different recordings resulting in a single video stream, called mashup. We start by eliciting requirements from focus groups, interviewing professional video editors and consulting film grammar literature. We design a formal model for automatic mashup generation based on maximizing the degree of fulfillment of the requirements. Various audio-visual content analysis techniques are used to determine how well the requirements are satisfied by a recording. To validate the system, we compare our mashups with two other mashups: manually created by a professional video editor and machine generated by random segment selection. The mashups are evaluated in terms of visual quality, content diversity and pleasantness by 40 subjects. The results show that our mashups and the manual mashups are perceived as comparable, while both of them are significantly higher than the random mashups in all three terms.

AB - A large number of videos are captured and shared by the audience from musical concerts. However, such recordings are typically perceived as boring mainly because of their limited view, poor visual quality and incomplete coverage. It is our objective to enrich the viewing experience of these recordings by exploiting the abundance of content from multiple sources. In this paper, we propose a novel Virtual Director system that automatically combines the most desirable segments from different recordings resulting in a single video stream, called mashup. We start by eliciting requirements from focus groups, interviewing professional video editors and consulting film grammar literature. We design a formal model for automatic mashup generation based on maximizing the degree of fulfillment of the requirements. Various audio-visual content analysis techniques are used to determine how well the requirements are satisfied by a recording. To validate the system, we compare our mashups with two other mashups: manually created by a professional video editor and machine generated by random segment selection. The mashups are evaluated in terms of visual quality, content diversity and pleasantness by 40 subjects. The results show that our mashups and the manual mashups are perceived as comparable, while both of them are significantly higher than the random mashups in all three terms.

U2 - 10.1145/1873951.1874023

DO - 10.1145/1873951.1874023

M3 - Other contribution

SN - 978-1-60558-933-6

PB - Association for Computing Machinery, Inc

ER -