Varying image description tasks: spoken versus written descriptions

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

98 Downloads (Pure)

Abstract

Automatic image description systems are commonly trained and evaluated on written image descriptions. At the same time, these systems are often used to provide spoken descriptions (e.g., for visually impaired users) through apps like TapTapSee or Seeing AI. This is not a problem, as long as spoken and written descriptions are very similar. However, linguistic research suggests that spoken language often differs from written language. These differences are not regular and vary from context to context. Therefore, this paper investigates whether there are differences between written and spoken image descriptions, even if they are elicited through similar tasks. We compared descriptions produced in two languages (English and Dutch) and found substantial differences between spoken and written descriptions in both languages. Future research should examine if users prefer the spoken over the written style and, if so, aim to emulate spoken descriptions.
Original languageEnglish
Title of host publicationProceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects
Pages88–100
Number of pages13
Publication statusPublished - 2018
Event5th Workshop on NLP for Similar Languages, Varieties and Dialects - Santa Fe, New Mexico, United States
Duration: 20 Aug 2018 → …

Workshop

Workshop5th Workshop on NLP for Similar Languages, Varieties and Dialects
Abbreviated titleVarDial
Country/TerritoryUnited States
CitySanta Fe, New Mexico
Period20/08/18 → …

Fingerprint

Dive into the research topics of 'Varying image description tasks: spoken versus written descriptions'. Together they form a unique fingerprint.

Cite this