DIDEC: The Dutch Image Description and Eye-tracking Corpus

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

54 Downloads (Pure)

Abstract

We present a corpus of spoken Dutch image descriptions, paired with two sets of eye-tracking data: free viewing, where participants look at images without any particular purpose, and description viewing, where we track eye movements while participants produce spoken descriptions of the images they are viewing. This paper describes the data collection procedure and the corpus itself, and provides an initial analysis of self-corrections in image descriptions. We also present two studies showing the potential of this data. Though these studies mainly serve as an example, we do find two interesting results: (1) the eye-tracking data for the description viewing task is more coherent than for the free-viewing task; (2) variation in image descriptions (also called image specificity; Jas and Parikh, 2015) is only moderately correlated across different languages. Our corpus can be used to gain a deeper understanding of the image description task, particularly how visual attention is correlated with the image description process.
Original languageEnglish
Title of host publicationProceedings of the 27th International Conference on Computational Linguistics
Pages3658–3669
Number of pages12
Publication statusPublished - 2018
EventInternational Conference on Computational Linguistics 2018 - Santa Fe Community Convention Center, Santa Fe, United States
Duration: 20 Aug 201826 Aug 2018
Conference number: 27
http://coling2018.org/

Conference

ConferenceInternational Conference on Computational Linguistics 2018
Abbreviated titleCOLING 2018
CountryUnited States
CitySanta Fe
Period20/08/1826/08/18
Internet address

Fingerprint

Eye movements

Cite this

van Miltenburg, E., Kadar, A., Koolen, R., & Krahmer, E. (2018). DIDEC: The Dutch Image Description and Eye-tracking Corpus. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 3658–3669)
van Miltenburg, Emiel ; Kadar, Akos ; Koolen, Ruud ; Krahmer, Emiel. / DIDEC: The Dutch Image Description and Eye-tracking Corpus. Proceedings of the 27th International Conference on Computational Linguistics. 2018. pp. 3658–3669
@inproceedings{2910d9ba0ee9494386316bd424809701,
title = "DIDEC: The Dutch Image Description and Eye-tracking Corpus",
abstract = "We present a corpus of spoken Dutch image descriptions, paired with two sets of eye-tracking data: free viewing, where participants look at images without any particular purpose, and description viewing, where we track eye movements while participants produce spoken descriptions of the images they are viewing. This paper describes the data collection procedure and the corpus itself, and provides an initial analysis of self-corrections in image descriptions. We also present two studies showing the potential of this data. Though these studies mainly serve as an example, we do find two interesting results: (1) the eye-tracking data for the description viewing task is more coherent than for the free-viewing task; (2) variation in image descriptions (also called image specificity; Jas and Parikh, 2015) is only moderately correlated across different languages. Our corpus can be used to gain a deeper understanding of the image description task, particularly how visual attention is correlated with the image description process.",
author = "{van Miltenburg}, Emiel and Akos Kadar and Ruud Koolen and Emiel Krahmer",
year = "2018",
language = "English",
pages = "3658–3669",
booktitle = "Proceedings of the 27th International Conference on Computational Linguistics",

}

van Miltenburg, E, Kadar, A, Koolen, R & Krahmer, E 2018, DIDEC: The Dutch Image Description and Eye-tracking Corpus. in Proceedings of the 27th International Conference on Computational Linguistics. pp. 3658–3669, International Conference on Computational Linguistics 2018, Santa Fe, United States, 20/08/18.

DIDEC: The Dutch Image Description and Eye-tracking Corpus. / van Miltenburg, Emiel; Kadar, Akos; Koolen, Ruud; Krahmer, Emiel.

Proceedings of the 27th International Conference on Computational Linguistics. 2018. p. 3658–3669.

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

TY - GEN

T1 - DIDEC: The Dutch Image Description and Eye-tracking Corpus

AU - van Miltenburg, Emiel

AU - Kadar, Akos

AU - Koolen, Ruud

AU - Krahmer, Emiel

PY - 2018

Y1 - 2018

N2 - We present a corpus of spoken Dutch image descriptions, paired with two sets of eye-tracking data: free viewing, where participants look at images without any particular purpose, and description viewing, where we track eye movements while participants produce spoken descriptions of the images they are viewing. This paper describes the data collection procedure and the corpus itself, and provides an initial analysis of self-corrections in image descriptions. We also present two studies showing the potential of this data. Though these studies mainly serve as an example, we do find two interesting results: (1) the eye-tracking data for the description viewing task is more coherent than for the free-viewing task; (2) variation in image descriptions (also called image specificity; Jas and Parikh, 2015) is only moderately correlated across different languages. Our corpus can be used to gain a deeper understanding of the image description task, particularly how visual attention is correlated with the image description process.

AB - We present a corpus of spoken Dutch image descriptions, paired with two sets of eye-tracking data: free viewing, where participants look at images without any particular purpose, and description viewing, where we track eye movements while participants produce spoken descriptions of the images they are viewing. This paper describes the data collection procedure and the corpus itself, and provides an initial analysis of self-corrections in image descriptions. We also present two studies showing the potential of this data. Though these studies mainly serve as an example, we do find two interesting results: (1) the eye-tracking data for the description viewing task is more coherent than for the free-viewing task; (2) variation in image descriptions (also called image specificity; Jas and Parikh, 2015) is only moderately correlated across different languages. Our corpus can be used to gain a deeper understanding of the image description task, particularly how visual attention is correlated with the image description process.

M3 - Conference contribution

SP - 3658

EP - 3669

BT - Proceedings of the 27th International Conference on Computational Linguistics

ER -

van Miltenburg E, Kadar A, Koolen R, Krahmer E. DIDEC: The Dutch Image Description and Eye-tracking Corpus. In Proceedings of the 27th International Conference on Computational Linguistics. 2018. p. 3658–3669